2024-03-29T03:58:37.900 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T03:58:37.900 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082 --verbose --owner scheduled_mchangir@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} --name mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi -- /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/orig.config.yaml 2024-03-29T03:58:37.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T03:58:38.034 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629082' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8207 sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBX/vqhpoVv7xQF9OaCd4qKW/a5NFdu1jYL3M9RLEAQtc36m1FhNG7RquWYaxP94P98XjDG4nPnlwf63rzrhEg= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVLz0WdmvAmWumf0aY3sfydbO2fS/7Cp/rwFGjG55ZgtF/39aOqwD8YdyUV7v2Ca0C0PHSlSZxhsByEx8VSW4g= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_02:01:30 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T03:58:38.035 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa; will attempt to use it 2024-03-29T03:58:38.035 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks 2024-03-29T03:58:38.035 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T03:58:38.037 INFO:teuthology.task.internal:Checking packages... 2024-03-29T03:58:38.061 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '69e2cd73c52284dfd5096c66b60c6e6b26a3c447' 2024-03-29T03:58:38.061 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T03:58:38.061 INFO:teuthology.packaging:ref: None 2024-03-29T03:58:38.061 INFO:teuthology.packaging:tag: None 2024-03-29T03:58:38.061 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:58:38.061 INFO:teuthology.packaging:sha1: 69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T03:58:38.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-29T03:58:38.200 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2520.g69e2cd73 2024-03-29T03:58:38.202 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T03:58:38.209 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T03:58:38.209 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T03:58:38.218 INFO:teuthology.task.internal:Saving configuration 2024-03-29T03:58:38.236 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T03:58:38.244 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T03:58:38.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:40.831617', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBX/vqhpoVv7xQF9OaCd4qKW/a5NFdu1jYL3M9RLEAQtc36m1FhNG7RquWYaxP94P98XjDG4nPnlwf63rzrhEg='} 2024-03-29T03:58:38.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:40.833252', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVLz0WdmvAmWumf0aY3sfydbO2fS/7Cp/rwFGjG55ZgtF/39aOqwD8YdyUV7v2Ca0C0PHSlSZxhsByEx8VSW4g='} 2024-03-29T03:58:38.293 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T03:58:38.301 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T03:58:38.301 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T03:58:38.301 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T03:58:38.377 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f20424e8f70>, signals=[15]) 2024-03-29T03:58:38.377 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T03:58:38.383 INFO:teuthology.task.internal:Opening connections... 2024-03-29T03:58:38.383 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T03:58:38.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:58:38.479 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T03:58:38.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T03:58:38.575 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T03:58:38.582 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-03-29T03:58:38.600 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-03-29T03:58:38.601 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-03-29T03:58:38.660 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:58:38.661 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:58:38.662 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:58:38.662 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-03-29T03:58:38.682 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-03-29T03:58:38.699 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-03-29T03:58:38.699 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-03-29T03:58:38.759 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-29T03:58:38.760 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-29T03:58:38.760 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-03-29T03:58:38.782 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T03:58:38.793 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T03:58:38.801 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T03:58:38.801 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:58:38.804 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-03-29T03:58:38.818 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T03:58:38.826 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T03:58:38.827 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:58:38.864 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-03-29T03:58:38.942 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T03:58:39.000 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T03:58:39.015 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-29T03:58:39.016 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-29T03:58:39.016 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-29T03:58:39.016 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:58:39.017 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T03:58:39.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:58:39.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T03:58:39.034 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-03-29T03:58:39.034 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-03-29T03:58:39.091 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:58:39.091 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.14.0-361.el9.x86_64 2024-03-29T03:58:39.091 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum install -y kernel 2024-03-29T03:58:39.094 INFO:teuthology.orchestra.run.smithi181.stdout:5.14.0-361.el9.x86_64 2024-03-29T03:58:39.094 INFO:teuthology.task.kernel:Running kernel on smithi181: 5.14.0-361.el9.x86_64 2024-03-29T03:58:39.094 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2024-03-29T03:58:41.160 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-29T03:58:41.284 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-29T03:58:42.576 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-29T03:58:42.667 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-03-29T03:58:43.301 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.0 MB 00:00 2024-03-29T03:58:43.874 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-03-29T03:58:45.246 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-29T03:58:45.976 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-03-29T03:58:52.784 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.1 MB 00:03 2024-03-29T03:58:53.169 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.1 MB 00:04 2024-03-29T03:58:54.259 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-03-29T03:58:54.794 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-29T03:58:56.269 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-29T03:58:56.915 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:01 2024-03-29T03:59:01.235 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-29T03:59:01.970 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-29T03:59:03.534 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:59:03.534 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:59:03.618 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-03-29T03:59:03.622 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout:Install 4 Packages 2024-03-29T03:59:03.623 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:03.624 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 76 M 2024-03-29T03:59:03.624 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 123 M 2024-03-29T03:59:03.624 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-03-29T03:59:03.786 INFO:teuthology.orchestra.run.smithi187.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-03-29T03:59:04.269 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-29T03:59:04.269 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-29T03:59:04.345 INFO:teuthology.orchestra.run.smithi187.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 21 MB/s | 15 MB 00:00 2024-03-29T03:59:04.357 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-03-29T03:59:04.362 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-03-29T03:59:04.362 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-03-29T03:59:04.362 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout:Install 4 Packages 2024-03-29T03:59:04.363 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:04.364 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 76 M 2024-03-29T03:59:04.364 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 123 M 2024-03-29T03:59:04.364 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-03-29T03:59:04.530 INFO:teuthology.orchestra.run.smithi181.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-29T03:59:04.995 INFO:teuthology.orchestra.run.smithi187.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-03-29T03:59:05.655 INFO:teuthology.orchestra.run.smithi181.stdout:(2/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 26 MB/s | 34 MB 00:01 2024-03-29T03:59:05.813 INFO:teuthology.orchestra.run.smithi181.stdout:(3/4): kernel-core-5.14.0-434.el9.x86_64.rpm 10 MB/s | 15 MB 00:01 2024-03-29T03:59:05.845 INFO:teuthology.orchestra.run.smithi187.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 13 MB/s | 28 MB 00:02 2024-03-29T03:59:05.848 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:59:05.848 INFO:teuthology.orchestra.run.smithi187.stdout:Total 34 MB/s | 76 MB 00:02 2024-03-29T03:59:05.848 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-03-29T03:59:06.218 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-03-29T03:59:06.218 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-03-29T03:59:06.397 INFO:teuthology.orchestra.run.smithi181.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 15 MB/s | 28 MB 00:01 2024-03-29T03:59:06.400 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-03-29T03:59:06.400 INFO:teuthology.orchestra.run.smithi181.stdout:Total 38 MB/s | 76 MB 00:02 2024-03-29T03:59:06.400 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-03-29T03:59:06.743 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-03-29T03:59:06.743 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-03-29T03:59:07.056 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-03-29T03:59:07.056 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-03-29T03:59:07.594 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-03-29T03:59:07.595 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-03-29T03:59:09.337 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-03-29T03:59:09.863 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-03-29T03:59:09.925 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:10.020 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:10.450 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:12.742 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:12.871 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:13.301 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:13.441 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:17.620 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:17.728 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:18.132 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:18.165 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:22.445 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:22.878 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:51.681 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:51.681 INFO:teuthology.orchestra.run.smithi187.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T03:59:51.681 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:51.725 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.053 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.053 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:52.053 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:52.053 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:52.177 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.177 INFO:teuthology.orchestra.run.smithi181.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T03:59:52.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:52.221 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T03:59:52.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:52.490 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-03-29T03:59:52.622 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:52.622 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-03-29T03:59:52.622 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-03-29T03:59:52.622 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-03-29T03:59:52.822 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:59:52.931 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:52.932 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:59:52.932 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T03:59:52.932 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T03:59:52.932 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T03:59:52.932 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-29T03:59:52.932 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T03:59:52.932 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:59:53.012 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.012 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.012 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.040 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.041 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.041 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:53.041 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-03-29T03:59:53.046 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2024-03-29T03:59:53.046 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.06 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 68.el9 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Group : Unspecified 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 8264254 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : builder@centos.org 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2024-03-29T03:59:53.047 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T03:59:53.048 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T03:59:53.049 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T03:59:53.049 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.049 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T03:59:53.369 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:59:53.480 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.480 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.480 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.480 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-29T03:59:53.480 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-29T03:59:53.480 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-29T03:59:53.480 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2024-03-29T03:59:53.480 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2024-03-29T03:59:53.560 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-5.14.0-434.el9.x86_64 2024-03-29T03:59:53.560 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.561 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2-tools 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2-tools 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.06 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 68.el9 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Group : Unspecified 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 8264254 2024-03-29T03:59:53.598 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : builder@centos.org 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : CentOS 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Support tools for GRUB. 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T03:59:53.599 INFO:teuthology.orchestra.run.smithi181.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T03:59:53.600 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T03:59:53.600 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 5.14.0-434.el9.x86_64 2024-03-29T03:59:53.600 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T03:59:53.722 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-03-29T03:59:54.209 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2024-03-29T03:59:55.019 INFO:teuthology.orchestra.run.smithi187.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T03:59:55.048 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-03-29T03:59:55.050 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-03-29T03:59:55.064 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.4Z70YwhoCW 2024-03-29T03:59:55.065 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4Z70YwhoCW 2024-03-29T03:59:55.132 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.4Z70YwhoCW 2024-03-29T03:59:55.267 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.4Z70YwhoCW is 6KB 2024-03-29T03:59:55.286 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.4Z70YwhoCW 2024-03-29T03:59:55.300 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T03:59:55.365 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T03:59:55.365 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T03:59:55.365 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T03:59:55.365 INFO:teuthology.orchestra.run.smithi187.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T03:59:55.366 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T03:59:55.510 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-03-29T03:59:55.644 INFO:teuthology.orchestra.run.smithi181.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-29T03:59:55.681 INFO:teuthology.orchestra.run.smithi181.stderr:done 2024-03-29T03:59:55.683 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-03-29T03:59:55.698 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.vvVhyAaZTo 2024-03-29T03:59:55.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vvVhyAaZTo 2024-03-29T03:59:55.767 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.vvVhyAaZTo 2024-03-29T03:59:55.913 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.vvVhyAaZTo is 6KB 2024-03-29T03:59:55.961 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.vvVhyAaZTo 2024-03-29T03:59:55.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T03:59:56.040 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-29T03:59:56.040 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-29T03:59:56.040 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-29T03:59:56.040 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-03-29T03:59:56.041 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-03-29T03:59:56.196 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2024-03-29T04:00:25.542 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-29T04:00:25.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-03-29T04:00:25.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:00:26.200 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-29T04:00:26.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-03-29T04:00:26.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:25.603 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T04:01:26.203 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T04:01:34.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-03-29T04:01:34.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:35.046 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-03-29T04:01:35.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-03-29T04:01:35.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:01:35.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-03-29T04:01:35.643 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:01:35.643 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-03-29T04:01:35.657 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:01:35.657 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:01:35.657 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:01:35.657 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:01:36.608 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-03-29T04:01:36.659 DEBUG:teuthology.parallel:result is None 2024-03-29T04:01:37.106 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-03-29T04:01:37.106 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-03-29T04:01:37.107 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-03-29T04:01:37.120 INFO:teuthology.orchestra.run.smithi181.stdout:5.14.0-434.el9.x86_64 2024-03-29T04:01:37.121 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-03-29T04:01:37.121 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T04:01:37.121 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-29T04:01:38.122 DEBUG:teuthology.parallel:result is None 2024-03-29T04:01:38.122 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T04:01:38.130 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T04:01:38.130 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:01:38.133 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T04:01:38.148 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T04:01:38.154 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T04:01:38.164 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T04:01:38.164 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:01:38.189 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T04:01:38.210 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T04:01:38.217 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T04:01:38.217 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:01:38.246 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T04:01:38.328 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:01:38.335 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:01:38.372 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:01:38.379 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T04:01:38.381 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T04:01:38.388 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T04:01:38.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:01:38.391 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T04:01:38.419 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T04:01:38.428 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T04:01:38.428 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:01:38.458 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T04:01:38.474 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:01:38.550 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:01:38.598 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:01:38.678 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:01:38.724 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:01:38.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:01:38.795 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:01:38.834 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T04:01:38.888 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:01:38.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T04:01:39.021 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:01:39.021 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T04:01:39.092 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-03-29T04:01:39.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-03-29T04:01:39.135 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:01:39.180 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:01:39.565 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T04:01:39.573 INFO:teuthology.task.internal:Starting timer... 2024-03-29T04:01:39.573 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T04:01:39.583 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T04:01:39.590 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-29T04:01:39.591 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-03-29T04:01:39.655 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-03-29T04:01:39.656 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-03-29T04:01:39.724 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-03-29T04:01:39.726 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T04:01:39.726 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-03-29T04:01:39.746 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-03-29T04:01:39.746 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-03-29T04:01:39.765 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-03-29T04:01:39.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-03-29T04:01:39.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:01:39.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:01:39.816 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:01:39.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:01:39.843 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T04:01:39.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:01:39.898 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-03-29T04:01:39.984 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T04:01:39.994 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-29T04:01:39.999 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-29T04:01:40.150 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T04:01:40.164 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T04:01:40.165 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi181.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T04:06:01.562 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-03-29T04:06:01.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-03-29T04:06:01.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:06:01.658 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-03-29T04:06:01.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-03-29T04:06:01.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-03-29T04:06:01.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T04:06:01.827 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-03-29T04:06:01.900 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-03-29T04:06:01.901 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T04:06:01.912 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T04:06:01.913 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:06:01.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:06:01.915 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T04:06:01.915 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:06:01.944 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:06:01.961 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:06:01.970 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T04:06:01.987 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T04:06:01.991 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-03-29T04:06:02.004 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-03-29T04:06:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-03-29T04:06:02.021 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:06:02.029 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-03-29T04:06:02.037 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:06:02.046 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T04:06:02.062 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T04:06:02.089 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:06:02.092 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:06:02.092 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-03-29T04:06:02.092 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.092 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.093 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.093 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.112 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:06:02.115 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:06:02.116 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-03-29T04:06:02.116 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.116 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.116 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.116 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:06:02.117 INFO:teuthology.run_tasks:Running task install... 2024-03-29T04:06:02.125 DEBUG:teuthology.task.install:project ceph 2024-03-29T04:06:02.125 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}} 2024-03-29T04:06:02.126 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'} 2024-03-29T04:06:02.126 INFO:teuthology.task.install:Using flavor: default 2024-03-29T04:06:02.131 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T04:06:02.132 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T04:06:02.132 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:06:02.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:06:02.134 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'tag': None, 'wait_for_package': False} 2024-03-29T04:06:02.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:06:02.280 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:06:02.280 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:06:02.348 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/ 2024-03-29T04:06:02.348 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2520.g69e2cd73 2024-03-29T04:06:02.389 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:06:02.389 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:06:02.389 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:06:02.417 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:06:02.418 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:06:02.463 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T04:06:02.464 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:06:02.464 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T04:06:02.485 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:06:02.492 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T04:06:02.492 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/69e2cd73c52284dfd5096c66b60c6e6b26a3c447/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T04:06:02.560 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T04:06:02.565 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:06:02.631 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-03-29T04:06:02.633 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-03-29T04:06:02.640 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T04:06:02.706 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-03-29T04:06:02.711 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-03-29T04:06:02.882 INFO:teuthology.orchestra.run.smithi181.stdout:74 files removed 2024-03-29T04:06:02.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:06:02.954 INFO:teuthology.orchestra.run.smithi187.stdout:74 files removed 2024-03-29T04:06:02.982 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T04:06:03.567 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 235 kB/s | 83 kB 00:00 2024-03-29T04:06:03.722 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 190 kB/s | 83 kB 00:00 2024-03-29T04:06:03.984 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-03-29T04:06:04.055 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-03-29T04:06:04.343 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-29T04:06:04.473 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-29T04:06:04.745 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-29T04:06:04.900 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-29T04:06:06.146 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-03-29T04:06:06.467 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-29T04:06:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 913 kB/s | 8.0 MB 00:09 2024-03-29T04:06:15.977 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 866 kB/s | 8.0 MB 00:09 2024-03-29T04:06:18.349 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-29T04:06:22.600 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.1 MB 00:00 2024-03-29T04:06:24.201 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-03-29T04:06:25.186 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-29T04:06:30.135 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-29T04:06:32.700 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:06:32.701 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:06:32.715 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:06:32.716 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:06:32.717 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:06:32.718 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:06:32.720 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:06:32.723 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:06:32.724 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:06:32.805 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:06:32.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:06:32.818 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:06:32.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:06:32.820 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:06:32.821 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:06:32.822 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:06:32.823 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:06:32.824 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:06:32.825 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:06:32.826 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:06:32.827 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:06:32.828 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:06:32.829 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout:Install 150 Packages 2024-03-29T04:06:32.830 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 24 Packages 2024-03-29T04:06:32.831 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:06:32.831 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 226 M 2024-03-29T04:06:32.831 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-03-29T04:06:33.686 INFO:teuthology.orchestra.run.smithi187.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-29T04:06:34.461 INFO:teuthology.orchestra.run.smithi187.stdout:(2/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-29T04:06:34.820 INFO:teuthology.orchestra.run.smithi187.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 404 kB/s | 145 kB 00:00 2024-03-29T04:06:34.895 INFO:teuthology.orchestra.run.smithi187.stdout:(4/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x8 3.4 MB/s | 5.4 MB 00:01 2024-03-29T04:06:35.253 INFO:teuthology.orchestra.run.smithi187.stdout:(5/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 4.2 MB/s | 1.5 MB 00:00 2024-03-29T04:06:36.170 INFO:teuthology.orchestra.run.smithi187.stdout:(6/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-03-29T04:06:36.287 INFO:teuthology.orchestra.run.smithi187.stdout:(7/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-03-29T04:06:37.934 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 926 kB/s | 19 MB 00:20 2024-03-29T04:06:38.998 INFO:teuthology.orchestra.run.smithi187.stdout:(8/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x86 6.0 MB/s | 17 MB 00:02 2024-03-29T04:06:39.039 INFO:teuthology.orchestra.run.smithi187.stdout:(9/174): ceph-selinux-19.0.0-2520.g69e2cd73.el9 610 kB/s | 25 kB 00:00 2024-03-29T04:06:40.665 INFO:teuthology.orchestra.run.smithi187.stdout:(10/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el 2.2 MB/s | 9.8 MB 00:04 2024-03-29T04:06:40.790 INFO:teuthology.orchestra.run.smithi187.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 1.7 MB/s | 221 kB 00:00 2024-03-29T04:06:40.856 INFO:teuthology.orchestra.run.smithi187.stdout:(12/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 406 kB/s | 27 kB 00:00 2024-03-29T04:06:40.924 INFO:teuthology.orchestra.run.smithi187.stdout:(13/174): libcephfs-devel-19.0.0-2520.g69e2cd73 476 kB/s | 32 kB 00:00 2024-03-29T04:06:41.149 INFO:teuthology.orchestra.run.smithi187.stdout:(14/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 3.2 MB/s | 746 kB 00:00 2024-03-29T04:06:41.257 INFO:teuthology.orchestra.run.smithi187.stdout:(15/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 1.5 MB/s | 163 kB 00:00 2024-03-29T04:06:41.316 INFO:teuthology.orchestra.run.smithi187.stdout:(16/174): librados-devel-19.0.0-2520.g69e2cd73. 2.1 MB/s | 127 kB 00:00 2024-03-29T04:06:41.500 INFO:teuthology.orchestra.run.smithi187.stdout:(17/174): libradosstriper1-19.0.0-2520.g69e2cd7 2.7 MB/s | 500 kB 00:00 2024-03-29T04:06:41.842 INFO:teuthology.orchestra.run.smithi187.stdout:(18/174): ceph-common-19.0.0-2520.g69e2cd73.el9 2.4 MB/s | 21 MB 00:08 2024-03-29T04:06:41.909 INFO:teuthology.orchestra.run.smithi187.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 680 kB/s | 45 kB 00:00 2024-03-29T04:06:42.034 INFO:teuthology.orchestra.run.smithi187.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 1.1 MB/s | 134 kB 00:00 2024-03-29T04:06:42.115 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.1 MB 00:00 2024-03-29T04:06:42.159 INFO:teuthology.orchestra.run.smithi187.stdout:(21/174): python3-cephfs-19.0.0-2520.g69e2cd73. 1.3 MB/s | 162 kB 00:00 2024-03-29T04:06:42.284 INFO:teuthology.orchestra.run.smithi187.stdout:(22/174): python3-rados-19.0.0-2520.g69e2cd73.e 2.5 MB/s | 321 kB 00:00 2024-03-29T04:06:42.410 INFO:teuthology.orchestra.run.smithi187.stdout:(23/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 2.3 MB/s | 298 kB 00:00 2024-03-29T04:06:42.485 INFO:teuthology.orchestra.run.smithi187.stdout:(24/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 1.3 MB/s | 99 kB 00:00 2024-03-29T04:06:42.543 INFO:teuthology.orchestra.run.smithi187.stdout:(25/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-29T04:06:43.085 INFO:teuthology.orchestra.run.smithi187.stdout:(26/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 3.2 MB/s | 5.1 MB 00:01 2024-03-29T04:06:43.160 INFO:teuthology.orchestra.run.smithi187.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-29T04:06:43.236 INFO:teuthology.orchestra.run.smithi187.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 452 kB/s | 34 kB 00:00 2024-03-29T04:06:43.311 INFO:teuthology.orchestra.run.smithi187.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 1.8 MB/s | 141 kB 00:00 2024-03-29T04:06:43.378 INFO:teuthology.orchestra.run.smithi187.stdout:(30/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 3.7 MB/s | 3.1 MB 00:00 2024-03-29T04:06:43.677 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-03-29T04:06:44.387 INFO:teuthology.orchestra.run.smithi187.stdout:(31/174): ceph-test-19.0.0-2520.g69e2cd73.el9.x 8.9 MB/s | 48 MB 00:05 2024-03-29T04:06:44.446 INFO:teuthology.orchestra.run.smithi187.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 3.1 MB/s | 3.5 MB 00:01 2024-03-29T04:06:44.470 INFO:teuthology.orchestra.run.smithi187.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2520.g69 2.9 MB/s | 249 kB 00:00 2024-03-29T04:06:44.496 INFO:teuthology.orchestra.run.smithi187.stdout:(34/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 988 kB/s | 49 kB 00:00 2024-03-29T04:06:44.521 INFO:teuthology.orchestra.run.smithi187.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2520.g6 329 kB/s | 16 kB 00:00 2024-03-29T04:06:44.571 INFO:teuthology.orchestra.run.smithi187.stdout:(36/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 3.4 MB/s | 264 kB 00:00 2024-03-29T04:06:44.638 INFO:teuthology.orchestra.run.smithi187.stdout:(37/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 6.4 MB/s | 762 kB 00:00 2024-03-29T04:06:44.664 INFO:teuthology.orchestra.run.smithi187.stdout:(38/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 198 kB/s | 18 kB 00:00 2024-03-29T04:06:44.847 INFO:teuthology.orchestra.run.smithi187.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-29T04:06:44.956 INFO:teuthology.orchestra.run.smithi187.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-03-29T04:06:44.998 INFO:teuthology.orchestra.run.smithi187.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-03-29T04:06:45.023 INFO:teuthology.orchestra.run.smithi187.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 604 kB/s | 106 kB 00:00 2024-03-29T04:06:45.048 INFO:teuthology.orchestra.run.smithi187.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-29T04:06:45.074 INFO:teuthology.orchestra.run.smithi187.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2024-03-29T04:06:45.224 INFO:teuthology.orchestra.run.smithi187.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-03-29T04:06:45.316 INFO:teuthology.orchestra.run.smithi187.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-03-29T04:06:45.333 INFO:teuthology.orchestra.run.smithi187.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 39 kB/s | 10 kB 00:00 2024-03-29T04:06:45.358 INFO:teuthology.orchestra.run.smithi187.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-03-29T04:06:45.377 INFO:teuthology.orchestra.run.smithi187.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:06:45.417 INFO:teuthology.orchestra.run.smithi187.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-29T04:06:45.445 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-29T04:06:45.450 INFO:teuthology.orchestra.run.smithi187.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-29T04:06:45.467 INFO:teuthology.orchestra.run.smithi187.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 507 kB/s | 46 kB 00:00 2024-03-29T04:06:45.492 INFO:teuthology.orchestra.run.smithi187.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-29T04:06:45.526 INFO:teuthology.orchestra.run.smithi187.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2024-03-29T04:06:45.551 INFO:teuthology.orchestra.run.smithi187.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-29T04:06:45.576 INFO:teuthology.orchestra.run.smithi187.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 408 kB/s | 34 kB 00:00 2024-03-29T04:06:45.735 INFO:teuthology.orchestra.run.smithi187.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-03-29T04:06:45.769 INFO:teuthology.orchestra.run.smithi187.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 4.2 MB/s | 817 kB 00:00 2024-03-29T04:06:45.802 INFO:teuthology.orchestra.run.smithi187.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 2024-03-29T04:06:45.828 INFO:teuthology.orchestra.run.smithi187.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-03-29T04:06:45.978 INFO:teuthology.orchestra.run.smithi187.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-03-29T04:06:46.003 INFO:teuthology.orchestra.run.smithi187.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 451 kB/s | 78 kB 00:00 2024-03-29T04:06:46.037 INFO:teuthology.orchestra.run.smithi187.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-03-29T04:06:46.145 INFO:teuthology.orchestra.run.smithi187.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-29T04:06:46.171 INFO:teuthology.orchestra.run.smithi187.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 150 kB/s | 24 kB 00:00 2024-03-29T04:06:46.196 INFO:teuthology.orchestra.run.smithi187.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-29T04:06:46.221 INFO:teuthology.orchestra.run.smithi187.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-29T04:06:46.246 INFO:teuthology.orchestra.run.smithi187.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-29T04:06:46.347 INFO:teuthology.orchestra.run.smithi187.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-29T04:06:46.389 INFO:teuthology.orchestra.run.smithi187.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-29T04:06:46.414 INFO:teuthology.orchestra.run.smithi187.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-29T04:06:46.439 INFO:teuthology.orchestra.run.smithi187.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 754 kB/s | 69 kB 00:00 2024-03-29T04:06:46.456 INFO:teuthology.orchestra.run.smithi187.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 991 kB/s | 41 kB 00:00 2024-03-29T04:06:46.481 INFO:teuthology.orchestra.run.smithi187.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-29T04:06:46.507 INFO:teuthology.orchestra.run.smithi187.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-29T04:06:46.532 INFO:teuthology.orchestra.run.smithi187.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-29T04:06:46.549 INFO:teuthology.orchestra.run.smithi187.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-03-29T04:06:46.574 INFO:teuthology.orchestra.run.smithi187.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 641 kB/s | 43 kB 00:00 2024-03-29T04:06:46.599 INFO:teuthology.orchestra.run.smithi187.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 775 kB/s | 39 kB 00:00 2024-03-29T04:06:46.716 INFO:teuthology.orchestra.run.smithi187.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 289 kB/s | 41 kB 00:00 2024-03-29T04:06:46.916 INFO:teuthology.orchestra.run.smithi187.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-29T04:06:46.942 INFO:teuthology.orchestra.run.smithi187.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 648 kB/s | 220 kB 00:00 2024-03-29T04:06:46.958 INFO:teuthology.orchestra.run.smithi187.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 145 kB/s | 35 kB 00:00 2024-03-29T04:06:46.984 INFO:teuthology.orchestra.run.smithi187.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 697 kB/s | 46 kB 00:00 2024-03-29T04:06:47.009 INFO:teuthology.orchestra.run.smithi187.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-29T04:06:47.034 INFO:teuthology.orchestra.run.smithi187.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-03-29T04:06:47.059 INFO:teuthology.orchestra.run.smithi187.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 833 kB/s | 63 kB 00:00 2024-03-29T04:06:47.085 INFO:teuthology.orchestra.run.smithi187.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-29T04:06:47.110 INFO:teuthology.orchestra.run.smithi187.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-29T04:06:47.135 INFO:teuthology.orchestra.run.smithi187.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-29T04:06:47.169 INFO:teuthology.orchestra.run.smithi187.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-29T04:06:47.194 INFO:teuthology.orchestra.run.smithi187.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-29T04:06:47.219 INFO:teuthology.orchestra.run.smithi187.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-29T04:06:47.244 INFO:teuthology.orchestra.run.smithi187.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-29T04:06:47.745 INFO:teuthology.orchestra.run.smithi187.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 391 kB/s | 195 kB 00:00 2024-03-29T04:06:47.770 INFO:teuthology.orchestra.run.smithi187.stdout:(96/174): jq-1.6-16.el9.x86_64.rpm 340 kB/s | 187 kB 00:00 2024-03-29T04:06:47.854 INFO:teuthology.orchestra.run.smithi187.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-29T04:06:47.925 INFO:teuthology.orchestra.run.smithi187.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-03-29T04:06:47.937 INFO:teuthology.orchestra.run.smithi187.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-03-29T04:06:47.996 INFO:teuthology.orchestra.run.smithi187.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-29T04:06:48.113 INFO:teuthology.orchestra.run.smithi187.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-03-29T04:06:48.172 INFO:teuthology.orchestra.run.smithi187.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-29T04:06:48.230 INFO:teuthology.orchestra.run.smithi187.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.0 MB/s | 248 kB 00:00 2024-03-29T04:06:48.281 INFO:teuthology.orchestra.run.smithi187.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-29T04:06:48.523 INFO:teuthology.orchestra.run.smithi187.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-03-29T04:06:48.581 INFO:teuthology.orchestra.run.smithi187.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 5.9 MB/s | 4.1 MB 00:00 2024-03-29T04:06:48.765 INFO:teuthology.orchestra.run.smithi187.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-29T04:06:48.898 INFO:teuthology.orchestra.run.smithi187.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-03-29T04:06:48.990 INFO:teuthology.orchestra.run.smithi187.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 661 kB/s | 309 kB 00:00 2024-03-29T04:06:49.016 INFO:teuthology.orchestra.run.smithi187.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-03-29T04:06:49.082 INFO:teuthology.orchestra.run.smithi187.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-03-29T04:06:49.108 INFO:teuthology.orchestra.run.smithi187.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-29T04:06:49.158 INFO:teuthology.orchestra.run.smithi187.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-03-29T04:06:49.183 INFO:teuthology.orchestra.run.smithi187.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-29T04:06:49.259 INFO:teuthology.orchestra.run.smithi187.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-03-29T04:06:49.284 INFO:teuthology.orchestra.run.smithi187.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-03-29T04:06:49.334 INFO:teuthology.orchestra.run.smithi187.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-03-29T04:06:49.359 INFO:teuthology.orchestra.run.smithi187.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-29T04:06:49.385 INFO:teuthology.orchestra.run.smithi187.stdout:(119/174): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-03-29T04:06:49.410 INFO:teuthology.orchestra.run.smithi187.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-29T04:06:49.444 INFO:teuthology.orchestra.run.smithi187.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-29T04:06:49.486 INFO:teuthology.orchestra.run.smithi187.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-03-29T04:06:49.511 INFO:teuthology.orchestra.run.smithi187.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-03-29T04:06:49.536 INFO:teuthology.orchestra.run.smithi187.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-29T04:06:49.586 INFO:teuthology.orchestra.run.smithi187.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-29T04:06:49.612 INFO:teuthology.orchestra.run.smithi187.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-03-29T04:06:49.654 INFO:teuthology.orchestra.run.smithi187.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-29T04:06:49.695 INFO:teuthology.orchestra.run.smithi187.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-03-29T04:06:49.712 INFO:teuthology.orchestra.run.smithi187.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-29T04:06:49.729 INFO:teuthology.orchestra.run.smithi187.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-29T04:06:49.772 INFO:teuthology.orchestra.run.smithi187.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-29T04:06:49.805 INFO:teuthology.orchestra.run.smithi187.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-03-29T04:06:49.847 INFO:teuthology.orchestra.run.smithi187.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-03-29T04:06:49.872 INFO:teuthology.orchestra.run.smithi187.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 785 kB/s | 79 kB 00:00 2024-03-29T04:06:49.898 INFO:teuthology.orchestra.run.smithi187.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-03-29T04:06:49.931 INFO:teuthology.orchestra.run.smithi187.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-29T04:06:49.957 INFO:teuthology.orchestra.run.smithi187.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-29T04:06:49.982 INFO:teuthology.orchestra.run.smithi187.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-29T04:06:50.007 INFO:teuthology.orchestra.run.smithi187.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-29T04:06:50.032 INFO:teuthology.orchestra.run.smithi187.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-29T04:06:50.058 INFO:teuthology.orchestra.run.smithi187.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-29T04:06:50.083 INFO:teuthology.orchestra.run.smithi187.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-03-29T04:06:50.100 INFO:teuthology.orchestra.run.smithi187.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-03-29T04:06:50.125 INFO:teuthology.orchestra.run.smithi187.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-03-29T04:06:50.159 INFO:teuthology.orchestra.run.smithi187.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-29T04:06:50.184 INFO:teuthology.orchestra.run.smithi187.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-29T04:06:50.209 INFO:teuthology.orchestra.run.smithi187.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-29T04:06:50.226 INFO:teuthology.orchestra.run.smithi187.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-03-29T04:06:50.251 INFO:teuthology.orchestra.run.smithi187.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-29T04:06:50.310 INFO:teuthology.orchestra.run.smithi187.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-29T04:06:50.345 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-29T04:06:50.452 INFO:teuthology.orchestra.run.smithi187.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 20 MB/s | 2.3 MB 00:00 2024-03-29T04:06:50.536 INFO:teuthology.orchestra.run.smithi187.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-29T04:06:50.569 INFO:teuthology.orchestra.run.smithi187.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-29T04:06:50.611 INFO:teuthology.orchestra.run.smithi187.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-29T04:06:50.661 INFO:teuthology.orchestra.run.smithi187.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-29T04:06:50.778 INFO:teuthology.orchestra.run.smithi187.stdout:(156/174): librados2-19.0.0-2520.g69e2cd73.el9. 6.2 MB/s | 3.4 MB 00:00 2024-03-29T04:06:50.929 INFO:teuthology.orchestra.run.smithi187.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 267 kB/s | 71 kB 00:00 2024-03-29T04:06:50.962 INFO:teuthology.orchestra.run.smithi187.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 198 kB/s | 36 kB 00:00 2024-03-29T04:06:51.046 INFO:teuthology.orchestra.run.smithi187.stdout:(159/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-03-29T04:06:51.230 INFO:teuthology.orchestra.run.smithi187.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 958 kB/s | 256 kB 00:00 2024-03-29T04:06:51.255 INFO:teuthology.orchestra.run.smithi187.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 110 kB/s | 23 kB 00:00 2024-03-29T04:06:51.280 INFO:teuthology.orchestra.run.smithi187.stdout:(162/174): elfutils-libelf-0.190-2.el9.x86_64.r 547 kB/s | 192 kB 00:00 2024-03-29T04:06:51.330 INFO:teuthology.orchestra.run.smithi187.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-29T04:06:51.389 INFO:teuthology.orchestra.run.smithi187.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-29T04:06:51.431 INFO:teuthology.orchestra.run.smithi187.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 637 kB/s | 64 kB 00:00 2024-03-29T04:06:51.514 INFO:teuthology.orchestra.run.smithi187.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 229 kB/s | 29 kB 00:00 2024-03-29T04:06:51.540 INFO:teuthology.orchestra.run.smithi187.stdout:(167/174): libfdisk-2.37.4-18.el9.x86_64.rpm 544 kB/s | 155 kB 00:00 2024-03-29T04:06:51.840 INFO:teuthology.orchestra.run.smithi187.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.0 MB/s | 1.2 MB 00:00 2024-03-29T04:06:51.948 INFO:teuthology.orchestra.run.smithi187.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-29T04:06:52.058 INFO:teuthology.orchestra.run.smithi187.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-29T04:06:52.099 INFO:teuthology.orchestra.run.smithi187.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 2.2 MB 00:00 2024-03-29T04:06:52.141 INFO:teuthology.orchestra.run.smithi187.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.8 MB/s | 2.3 MB 00:00 2024-03-29T04:06:52.175 INFO:teuthology.orchestra.run.smithi187.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-03-29T04:06:52.200 INFO:teuthology.orchestra.run.smithi187.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 741 kB/s | 74 kB 00:00 2024-03-29T04:06:52.208 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:06:52.208 INFO:teuthology.orchestra.run.smithi187.stdout:Total 12 MB/s | 226 MB 00:19 2024-03-29T04:06:52.576 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-03-29T04:06:52.854 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-03-29T04:06:52.854 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-03-29T04:06:52.917 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:06:52.918 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-29T04:06:52.933 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-29T04:06:52.934 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-29T04:06:52.935 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-29T04:06:52.936 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-29T04:06:52.938 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-29T04:06:52.941 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-29T04:06:52.943 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-29T04:06:53.027 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 6.6 k 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.4 M 2024-03-29T04:06:53.057 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 890 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 145 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 1.5 M 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 141 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 3.5 M 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 7.4 M 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 49 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 9.8 M 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 48 M 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 264 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 762 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 221 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 27 k 2024-03-29T04:06:53.058 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 18 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 32 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 746 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 127 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 162 k 2024-03-29T04:06:53.059 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 321 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 298 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 99 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 85 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 170 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-03-29T04:06:53.060 INFO:teuthology.orchestra.run.smithi181.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.4 M 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 3.1 M 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-29T04:06:53.061 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-29T04:06:53.062 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 21 M 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 34 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 2.4 M 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 249 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 4.7 M 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 17 M 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2520.g69e2cd73.el9 ceph-noarch 16 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 25 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-29T04:06:53.063 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 163 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-29T04:06:53.064 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 500 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 5.1 M 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-29T04:06:53.065 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-29T04:06:53.066 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 45 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.0.0-2520.g69e2cd73.el9 ceph 134 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-29T04:06:53.067 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-29T04:06:53.068 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-29T04:06:53.069 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-29T04:06:53.070 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-29T04:06:53.071 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:Install 150 Packages 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 24 Packages 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:06:53.072 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 226 M 2024-03-29T04:06:53.073 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-03-29T04:06:53.755 INFO:teuthology.orchestra.run.smithi181.stdout:(1/174): ceph-19.0.0-2520.g69e2cd73.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-03-29T04:06:53.971 INFO:teuthology.orchestra.run.smithi181.stdout:(2/174): ceph-fuse-19.0.0-2520.g69e2cd73.el9.x8 4.0 MB/s | 890 kB 00:00 2024-03-29T04:06:54.022 INFO:teuthology.orchestra.run.smithi181.stdout:(3/174): ceph-immutable-object-cache-19.0.0-252 2.8 MB/s | 145 kB 00:00 2024-03-29T04:06:54.282 INFO:teuthology.orchestra.run.smithi181.stdout:(4/174): ceph-mds-19.0.0-2520.g69e2cd73.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-03-29T04:06:54.464 INFO:teuthology.orchestra.run.smithi181.stdout:(5/174): ceph-mgr-19.0.0-2520.g69e2cd73.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-03-29T04:06:54.573 INFO:teuthology.orchestra.run.smithi181.stdout:(6/174): ceph-base-19.0.0-2520.g69e2cd73.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-03-29T04:06:54.846 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-03-29T04:06:54.847 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-03-29T04:06:54.931 INFO:teuthology.orchestra.run.smithi181.stdout:(7/174): ceph-mon-19.0.0-2520.g69e2cd73.el9.x86 10 MB/s | 4.7 MB 00:00 2024-03-29T04:06:55.990 INFO:teuthology.orchestra.run.smithi181.stdout:(8/174): ceph-common-19.0.0-2520.g69e2cd73.el9. 8.6 MB/s | 21 MB 00:02 2024-03-29T04:06:56.115 INFO:teuthology.orchestra.run.smithi181.stdout:(9/174): ceph-radosgw-19.0.0-2520.g69e2cd73.el9 8.3 MB/s | 9.8 MB 00:01 2024-03-29T04:06:56.140 INFO:teuthology.orchestra.run.smithi181.stdout:(10/174): ceph-selinux-19.0.0-2520.g69e2cd73.el 168 kB/s | 25 kB 00:00 2024-03-29T04:06:56.216 INFO:teuthology.orchestra.run.smithi181.stdout:(11/174): cephfs-mirror-19.0.0-2520.g69e2cd73.e 2.9 MB/s | 221 kB 00:00 2024-03-29T04:06:56.291 INFO:teuthology.orchestra.run.smithi181.stdout:(12/174): cephfs-shell-19.0.0-2520.g69e2cd73.el 361 kB/s | 27 kB 00:00 2024-03-29T04:06:56.366 INFO:teuthology.orchestra.run.smithi181.stdout:(13/174): libcephfs-devel-19.0.0-2520.g69e2cd73 424 kB/s | 32 kB 00:00 2024-03-29T04:06:56.650 INFO:teuthology.orchestra.run.smithi181.stdout:(14/174): ceph-osd-19.0.0-2520.g69e2cd73.el9.x8 8.1 MB/s | 17 MB 00:02 2024-03-29T04:06:56.704 INFO:teuthology.orchestra.run.smithi181.stdout:(15/174): libcephfs2-19.0.0-2520.g69e2cd73.el9. 2.2 MB/s | 746 kB 00:00 2024-03-29T04:06:56.733 INFO:teuthology.orchestra.run.smithi181.stdout:(16/174): libcephsqlite-19.0.0-2520.g69e2cd73.e 1.9 MB/s | 163 kB 00:00 2024-03-29T04:06:56.784 INFO:teuthology.orchestra.run.smithi181.stdout:(17/174): librados-devel-19.0.0-2520.g69e2cd73. 1.6 MB/s | 127 kB 00:00 2024-03-29T04:06:56.817 INFO:teuthology.orchestra.run.smithi181.stdout:(18/174): libradosstriper1-19.0.0-2520.g69e2cd7 5.8 MB/s | 500 kB 00:00 2024-03-29T04:06:56.884 INFO:teuthology.orchestra.run.smithi181.stdout:(19/174): python3-ceph-argparse-19.0.0-2520.g69 679 kB/s | 45 kB 00:00 2024-03-29T04:06:56.976 INFO:teuthology.orchestra.run.smithi181.stdout:(20/174): python3-ceph-common-19.0.0-2520.g69e2 1.4 MB/s | 134 kB 00:00 2024-03-29T04:06:57.060 INFO:teuthology.orchestra.run.smithi181.stdout:(21/174): python3-cephfs-19.0.0-2520.g69e2cd73. 1.9 MB/s | 162 kB 00:00 2024-03-29T04:06:57.160 INFO:teuthology.orchestra.run.smithi181.stdout:(22/174): python3-rados-19.0.0-2520.g69e2cd73.e 3.1 MB/s | 321 kB 00:00 2024-03-29T04:06:57.260 INFO:teuthology.orchestra.run.smithi181.stdout:(23/174): python3-rbd-19.0.0-2520.g69e2cd73.el9 2.9 MB/s | 298 kB 00:00 2024-03-29T04:06:57.336 INFO:teuthology.orchestra.run.smithi181.stdout:(24/174): librgw2-19.0.0-2520.g69e2cd73.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-29T04:06:57.362 INFO:teuthology.orchestra.run.smithi181.stdout:(25/174): python3-rgw-19.0.0-2520.g69e2cd73.el9 991 kB/s | 99 kB 00:00 2024-03-29T04:06:57.403 INFO:teuthology.orchestra.run.smithi181.stdout:(26/174): rbd-fuse-19.0.0-2520.g69e2cd73.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-29T04:06:57.495 INFO:teuthology.orchestra.run.smithi181.stdout:(27/174): rbd-nbd-19.0.0-2520.g69e2cd73.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-29T04:06:57.595 INFO:teuthology.orchestra.run.smithi181.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2520.g 339 kB/s | 34 kB 00:00 2024-03-29T04:06:57.696 INFO:teuthology.orchestra.run.smithi181.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2520.g69e2cd7 1.4 MB/s | 141 kB 00:00 2024-03-29T04:06:57.746 INFO:teuthology.orchestra.run.smithi181.stdout:(30/174): rbd-mirror-19.0.0-2520.g69e2cd73.el9. 8.0 MB/s | 3.1 MB 00:00 2024-03-29T04:06:58.138 INFO:teuthology.orchestra.run.smithi181.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2520.g69e2c 8.0 MB/s | 3.5 MB 00:00 2024-03-29T04:06:58.238 INFO:teuthology.orchestra.run.smithi181.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2520.g69 2.4 MB/s | 249 kB 00:00 2024-03-29T04:06:58.347 INFO:teuthology.orchestra.run.smithi181.stdout:(33/174): ceph-mgr-rook-19.0.0-2520.g69e2cd73.e 456 kB/s | 49 kB 00:00 2024-03-29T04:06:58.447 INFO:teuthology.orchestra.run.smithi181.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2520.g6 164 kB/s | 16 kB 00:00 2024-03-29T04:06:58.556 INFO:teuthology.orchestra.run.smithi181.stdout:(35/174): ceph-volume-19.0.0-2520.g69e2cd73.el9 2.4 MB/s | 264 kB 00:00 2024-03-29T04:06:58.856 INFO:teuthology.orchestra.run.smithi181.stdout:(36/174): cephadm-19.0.0-2520.g69e2cd73.el9.noa 2.5 MB/s | 762 kB 00:00 2024-03-29T04:06:58.948 INFO:teuthology.orchestra.run.smithi181.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-03-29T04:06:58.973 INFO:teuthology.orchestra.run.smithi181.stdout:(38/174): cephfs-top-19.0.0-2520.g69e2cd73.el9. 156 kB/s | 18 kB 00:00 2024-03-29T04:06:59.123 INFO:teuthology.orchestra.run.smithi181.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-29T04:06:59.165 INFO:teuthology.orchestra.run.smithi181.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2024-03-29T04:06:59.191 INFO:teuthology.orchestra.run.smithi181.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-29T04:06:59.216 INFO:teuthology.orchestra.run.smithi181.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-29T04:06:59.233 INFO:teuthology.orchestra.run.smithi181.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 765 kB/s | 32 kB 00:00 2024-03-29T04:06:59.333 INFO:teuthology.orchestra.run.smithi181.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-29T04:06:59.350 INFO:teuthology.orchestra.run.smithi181.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 17 kB 00:00 2024-03-29T04:06:59.375 INFO:teuthology.orchestra.run.smithi181.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 402 kB/s | 10 kB 00:00 2024-03-29T04:06:59.401 INFO:teuthology.orchestra.run.smithi181.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-29T04:06:59.418 INFO:teuthology.orchestra.run.smithi181.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:06:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-29T04:06:59.468 INFO:teuthology.orchestra.run.smithi181.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-29T04:06:59.493 INFO:teuthology.orchestra.run.smithi181.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2024-03-29T04:06:59.519 INFO:teuthology.orchestra.run.smithi181.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-29T04:06:59.544 INFO:teuthology.orchestra.run.smithi181.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-29T04:06:59.569 INFO:teuthology.orchestra.run.smithi181.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-29T04:06:59.595 INFO:teuthology.orchestra.run.smithi181.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-29T04:06:59.712 INFO:teuthology.orchestra.run.smithi181.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-29T04:06:59.745 INFO:teuthology.orchestra.run.smithi181.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-03-29T04:06:59.779 INFO:teuthology.orchestra.run.smithi181.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-03-29T04:06:59.805 INFO:teuthology.orchestra.run.smithi181.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-03-29T04:06:59.930 INFO:teuthology.orchestra.run.smithi181.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-29T04:06:59.963 INFO:teuthology.orchestra.run.smithi181.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-29T04:06:59.988 INFO:teuthology.orchestra.run.smithi181.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-29T04:07:00.097 INFO:teuthology.orchestra.run.smithi181.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 43 MB/s | 5.8 MB 00:00 2024-03-29T04:07:00.122 INFO:teuthology.orchestra.run.smithi181.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 185 kB/s | 24 kB 00:00 2024-03-29T04:07:00.148 INFO:teuthology.orchestra.run.smithi181.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 363 kB/s | 18 kB 00:00 2024-03-29T04:07:00.173 INFO:teuthology.orchestra.run.smithi181.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-29T04:07:00.198 INFO:teuthology.orchestra.run.smithi181.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-29T04:07:00.299 INFO:teuthology.orchestra.run.smithi181.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-29T04:07:00.349 INFO:teuthology.orchestra.run.smithi181.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-29T04:07:00.374 INFO:teuthology.orchestra.run.smithi181.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-03-29T04:07:00.400 INFO:teuthology.orchestra.run.smithi181.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-29T04:07:00.425 INFO:teuthology.orchestra.run.smithi181.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 820 kB/s | 41 kB 00:00 2024-03-29T04:07:00.450 INFO:teuthology.orchestra.run.smithi181.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-29T04:07:00.476 INFO:teuthology.orchestra.run.smithi181.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-29T04:07:00.501 INFO:teuthology.orchestra.run.smithi181.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-29T04:07:00.526 INFO:teuthology.orchestra.run.smithi181.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 948 kB/s | 47 kB 00:00 2024-03-29T04:07:00.551 INFO:teuthology.orchestra.run.smithi181.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-03-29T04:07:00.710 INFO:teuthology.orchestra.run.smithi181.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 247 kB/s | 39 kB 00:00 2024-03-29T04:07:00.860 INFO:teuthology.orchestra.run.smithi181.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-29T04:07:00.885 INFO:teuthology.orchestra.run.smithi181.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 236 kB/s | 41 kB 00:00 2024-03-29T04:07:00.911 INFO:teuthology.orchestra.run.smithi181.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-03-29T04:07:00.928 INFO:teuthology.orchestra.run.smithi181.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 840 kB/s | 35 kB 00:00 2024-03-29T04:07:00.944 INFO:teuthology.orchestra.run.smithi181.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.3 MB/s | 46 kB 00:00 2024-03-29T04:07:00.970 INFO:teuthology.orchestra.run.smithi181.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-29T04:07:00.987 INFO:teuthology.orchestra.run.smithi181.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-29T04:07:01.004 INFO:teuthology.orchestra.run.smithi181.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-29T04:07:01.029 INFO:teuthology.orchestra.run.smithi181.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-29T04:07:01.055 INFO:teuthology.orchestra.run.smithi181.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-29T04:07:01.080 INFO:teuthology.orchestra.run.smithi181.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-29T04:07:01.114 INFO:teuthology.orchestra.run.smithi181.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-29T04:07:01.139 INFO:teuthology.orchestra.run.smithi181.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-29T04:07:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-29T04:07:01.190 INFO:teuthology.orchestra.run.smithi181.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-29T04:07:01.632 INFO:teuthology.orchestra.run.smithi181.stdout:(94/174): ceph-test-19.0.0-2520.g69e2cd73.el9.x 8.6 MB/s | 48 MB 00:05 2024-03-29T04:07:01.740 INFO:teuthology.orchestra.run.smithi181.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 325 kB/s | 187 kB 00:00 2024-03-29T04:07:01.781 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-03-29T04:07:01.782 INFO:teuthology.orchestra.run.smithi181.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-29T04:07:01.807 INFO:teuthology.orchestra.run.smithi181.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-29T04:07:01.871 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:07:01.906 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:07:01.908 INFO:teuthology.orchestra.run.smithi181.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-03-29T04:07:01.985 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:01.994 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:02.008 INFO:teuthology.orchestra.run.smithi181.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 95 kB/s | 21 kB 00:00 2024-03-29T04:07:02.033 INFO:teuthology.orchestra.run.smithi181.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 113 kB/s | 25 kB 00:00 2024-03-29T04:07:02.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:07:02.332 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:07:02.373 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:07:02.398 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:07:02.448 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:07:02.492 INFO:teuthology.orchestra.run.smithi181.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 507 kB/s | 245 kB 00:00 2024-03-29T04:07:02.517 INFO:teuthology.orchestra.run.smithi181.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 513 kB/s | 248 kB 00:00 2024-03-29T04:07:02.559 INFO:teuthology.orchestra.run.smithi181.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 540 kB/s | 22 kB 00:00 2024-03-29T04:07:02.709 INFO:teuthology.orchestra.run.smithi181.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-03-29T04:07:02.796 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:07:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:02.876 INFO:teuthology.orchestra.run.smithi181.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 244 kB/s | 94 kB 00:00 2024-03-29T04:07:02.965 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:02.985 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:03.076 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:03.106 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:07:03.149 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:07:03.176 INFO:teuthology.orchestra.run.smithi181.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-29T04:07:03.251 INFO:teuthology.orchestra.run.smithi181.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-03-29T04:07:03.294 INFO:teuthology.orchestra.run.smithi181.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 740 kB/s | 309 kB 00:00 2024-03-29T04:07:03.334 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:07:03.369 INFO:teuthology.orchestra.run.smithi181.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-03-29T04:07:03.387 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:07:03.415 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:03.461 INFO:teuthology.orchestra.run.smithi181.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-03-29T04:07:03.518 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:03.520 INFO:teuthology.orchestra.run.smithi181.stdout:(111/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.6 MB/s | 4.1 MB 00:01 2024-03-29T04:07:03.541 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:03.545 INFO:teuthology.orchestra.run.smithi181.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-03-29T04:07:03.596 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:03.612 INFO:teuthology.orchestra.run.smithi181.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-29T04:07:03.645 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:07:03.668 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:03.683 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:03.729 INFO:teuthology.orchestra.run.smithi181.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-29T04:07:03.754 INFO:teuthology.orchestra.run.smithi181.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-29T04:07:03.796 INFO:teuthology.orchestra.run.smithi181.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-03-29T04:07:03.821 INFO:teuthology.orchestra.run.smithi181.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-03-29T04:07:03.861 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:03.863 INFO:teuthology.orchestra.run.smithi181.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-03-29T04:07:03.896 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:03.896 INFO:teuthology.orchestra.run.smithi181.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-29T04:07:03.930 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:07:03.939 INFO:teuthology.orchestra.run.smithi181.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-29T04:07:03.964 INFO:teuthology.orchestra.run.smithi181.stdout:(121/174): luarocks-3.9.2-1.el9.noarch.rpm 340 kB/s | 151 kB 00:00 2024-03-29T04:07:04.023 INFO:teuthology.orchestra.run.smithi181.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-29T04:07:04.048 INFO:teuthology.orchestra.run.smithi181.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-03-29T04:07:04.093 INFO:teuthology.orchestra.run.smithi181.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-03-29T04:07:04.118 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:07:04.125 INFO:teuthology.orchestra.run.smithi181.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-29T04:07:04.166 INFO:teuthology.orchestra.run.smithi181.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 148 kB/s | 11 kB 00:00 2024-03-29T04:07:04.208 INFO:teuthology.orchestra.run.smithi181.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-29T04:07:04.232 INFO:teuthology.orchestra.run.smithi181.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-03-29T04:07:04.244 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:04.258 INFO:teuthology.orchestra.run.smithi181.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-03-29T04:07:04.283 INFO:teuthology.orchestra.run.smithi181.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-29T04:07:04.287 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:04.308 INFO:teuthology.orchestra.run.smithi181.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-29T04:07:04.359 INFO:teuthology.orchestra.run.smithi181.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-03-29T04:07:04.384 INFO:teuthology.orchestra.run.smithi181.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-29T04:07:04.386 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:04.434 INFO:teuthology.orchestra.run.smithi181.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-03-29T04:07:04.439 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:04.468 INFO:teuthology.orchestra.run.smithi181.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-03-29T04:07:04.493 INFO:teuthology.orchestra.run.smithi181.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-29T04:07:04.518 INFO:teuthology.orchestra.run.smithi181.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-29T04:07:04.544 INFO:teuthology.orchestra.run.smithi181.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-29T04:07:04.560 INFO:teuthology.orchestra.run.smithi181.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-03-29T04:07:04.594 INFO:teuthology.orchestra.run.smithi181.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-03-29T04:07:04.620 INFO:teuthology.orchestra.run.smithi181.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-29T04:07:04.636 INFO:teuthology.orchestra.run.smithi181.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-03-29T04:07:04.642 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:04.662 INFO:teuthology.orchestra.run.smithi181.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-03-29T04:07:04.687 INFO:teuthology.orchestra.run.smithi181.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 757 kB/s | 51 kB 00:00 2024-03-29T04:07:04.712 INFO:teuthology.orchestra.run.smithi181.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-29T04:07:04.746 INFO:teuthology.orchestra.run.smithi181.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-29T04:07:04.753 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:07:04.771 INFO:teuthology.orchestra.run.smithi181.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-29T04:07:04.788 INFO:teuthology.orchestra.run.smithi181.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 263 kB/s | 20 kB 00:00 2024-03-29T04:07:04.822 INFO:teuthology.orchestra.run.smithi181.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-29T04:07:04.848 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:07:04.878 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:07:04.897 INFO:teuthology.orchestra.run.smithi181.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-29T04:07:04.904 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:07:04.940 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:07:04.973 INFO:teuthology.orchestra.run.smithi181.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 31 MB/s | 2.3 MB 00:00 2024-03-29T04:07:05.005 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:07:05.039 INFO:teuthology.orchestra.run.smithi181.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-03-29T04:07:05.059 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:07:05.065 INFO:teuthology.orchestra.run.smithi181.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-29T04:07:05.098 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:07:05.107 INFO:teuthology.orchestra.run.smithi181.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-29T04:07:05.149 INFO:teuthology.orchestra.run.smithi181.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-29T04:07:05.161 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:07:05.197 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:07:05.241 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:07:05.265 INFO:teuthology.orchestra.run.smithi181.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 611 kB/s | 71 kB 00:00 2024-03-29T04:07:05.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:07:05.299 INFO:teuthology.orchestra.run.smithi181.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-03-29T04:07:05.301 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:07:05.358 INFO:teuthology.orchestra.run.smithi181.stdout:(158/174): librados2-19.0.0-2520.g69e2cd73.el9. 6.0 MB/s | 3.4 MB 00:00 2024-03-29T04:07:05.370 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:07:05.383 INFO:teuthology.orchestra.run.smithi181.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.2 MB/s | 192 kB 00:00 2024-03-29T04:07:05.405 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:07:05.425 INFO:teuthology.orchestra.run.smithi181.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-03-29T04:07:05.449 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:07:05.475 INFO:teuthology.orchestra.run.smithi181.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-29T04:07:05.478 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:07:05.500 INFO:teuthology.orchestra.run.smithi181.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-29T04:07:05.526 INFO:teuthology.orchestra.run.smithi181.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-29T04:07:05.546 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:07:05.551 INFO:teuthology.orchestra.run.smithi181.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-29T04:07:05.576 INFO:teuthology.orchestra.run.smithi181.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-29T04:07:05.592 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:07:05.601 INFO:teuthology.orchestra.run.smithi181.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-03-29T04:07:05.666 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:07:05.669 INFO:teuthology.orchestra.run.smithi181.stdout:(167/174): librbd1-19.0.0-2520.g69e2cd73.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-03-29T04:07:05.719 INFO:teuthology.orchestra.run.smithi181.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-29T04:07:05.799 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:07:05.803 INFO:teuthology.orchestra.run.smithi181.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 5.5 MB/s | 465 kB 00:00 2024-03-29T04:07:05.853 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:07:05.869 INFO:teuthology.orchestra.run.smithi181.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 2.2 MB 00:00 2024-03-29T04:07:05.895 INFO:teuthology.orchestra.run.smithi181.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-03-29T04:07:06.012 INFO:teuthology.orchestra.run.smithi181.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.7 MB/s | 2.3 MB 00:00 2024-03-29T04:07:06.129 INFO:teuthology.orchestra.run.smithi181.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.1 MB/s | 1.1 MB 00:00 2024-03-29T04:07:06.187 INFO:teuthology.orchestra.run.smithi181.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 255 kB/s | 74 kB 00:00 2024-03-29T04:07:06.193 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-03-29T04:07:06.194 INFO:teuthology.orchestra.run.smithi181.stdout:Total 17 MB/s | 226 MB 00:13 2024-03-29T04:07:06.369 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:07:06.448 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:07:06.524 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:07:06.557 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-03-29T04:07:06.585 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:06.595 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:06.635 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:07:06.675 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:07:06.704 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:07:06.735 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:07:06.831 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-03-29T04:07:06.832 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-03-29T04:07:07.005 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:07:07.211 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:07:07.295 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:07:07.338 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:07:07.486 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:07:07.551 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:07:07.610 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:07:07.677 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:07:07.709 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:07:07.746 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:07:07.792 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:07:07.828 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:07:07.871 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:07:07.913 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:07:08.047 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:07:08.088 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:07:08.119 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:07:08.160 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:07:08.253 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:07:08.293 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:07:08.333 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:07:08.359 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:07:08.383 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:07:08.462 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:07:08.584 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:07:08.691 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:07:08.741 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:07:08.790 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:07:08.827 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-03-29T04:07:08.827 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-03-29T04:07:08.846 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:07:09.028 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:07:09.079 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:07:09.206 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:07:09.244 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:07:09.276 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:07:09.307 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:07:09.335 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:07:09.360 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:07:09.400 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:07:09.431 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:07:09.461 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:07:09.554 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:07:09.588 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:07:09.621 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:07:09.663 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:07:09.698 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:07:09.777 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:07:09.859 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:07:09.903 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:07:09.935 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:07:09.982 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:07:10.078 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:07:10.128 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:07:10.853 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:07:10.900 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:07:10.933 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:07:10.982 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:07:11.060 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:07:11.109 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:07:11.287 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:07:11.316 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:07:11.394 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:07:12.031 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:07:12.136 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:07:13.359 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:07:13.457 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:07:13.488 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:07:13.519 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:07:13.547 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:07:13.859 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:07:13.885 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:07:13.959 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:07:13.990 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:07:14.026 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:07:14.478 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:07:14.505 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:07:14.559 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:07:14.581 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:07:15.745 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-03-29T04:07:15.834 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-29T04:07:15.860 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-29T04:07:15.906 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:15.916 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-29T04:07:15.959 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-29T04:07:16.027 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-29T04:07:16.059 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-29T04:07:16.085 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-29T04:07:16.135 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-29T04:07:16.466 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-29T04:07:16.520 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:16.642 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:16.671 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:07:16.812 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-29T04:07:16.853 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:17.019 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-29T04:07:17.073 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-29T04:07:17.101 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:17.188 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:17.211 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:17.259 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 17/198 2024-03-29T04:07:17.298 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-29T04:07:17.321 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:17.336 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-29T04:07:17.401 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:17.439 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:07:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:17.528 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:07:17.528 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:07:17.528 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:07:17.528 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:17.530 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-29T04:07:17.561 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-29T04:07:17.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:07:17.725 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-29T04:07:17.863 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:17.923 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-29T04:07:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:18.271 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-29T04:07:18.381 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-29T04:07:18.468 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-29T04:07:18.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-29T04:07:18.526 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-29T04:07:18.572 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-29T04:07:18.660 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-29T04:07:18.706 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-29T04:07:18.744 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-29T04:07:18.808 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-29T04:07:18.843 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-29T04:07:18.886 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-29T04:07:18.919 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-29T04:07:18.947 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-29T04:07:19.016 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-29T04:07:19.060 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 39/198 2024-03-29T04:07:19.095 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 40/198 2024-03-29T04:07:19.125 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 41/198 2024-03-29T04:07:19.193 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-29T04:07:19.238 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-29T04:07:19.312 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-29T04:07:19.444 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-29T04:07:19.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-29T04:07:20.009 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-29T04:07:20.086 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-29T04:07:20.182 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-29T04:07:20.223 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:20.233 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-29T04:07:20.272 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-29T04:07:20.314 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-29T04:07:20.341 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 53/198 2024-03-29T04:07:20.372 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-29T04:07:20.646 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-29T04:07:20.838 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-29T04:07:20.957 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-29T04:07:21.221 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-29T04:07:21.387 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-29T04:07:21.430 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-29T04:07:21.485 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-29T04:07:21.557 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-29T04:07:21.596 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-29T04:07:21.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-29T04:07:21.679 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-29T04:07:21.715 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-29T04:07:21.758 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-29T04:07:21.800 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-29T04:07:21.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-29T04:07:21.998 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-29T04:07:22.028 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-29T04:07:22.069 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-29T04:07:22.163 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-29T04:07:22.194 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-29T04:07:22.247 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-29T04:07:22.277 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-29T04:07:22.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-29T04:07:22.380 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-29T04:07:22.511 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-29T04:07:22.627 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-29T04:07:22.668 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-29T04:07:22.717 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-29T04:07:22.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-29T04:07:22.955 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-29T04:07:22.997 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-29T04:07:23.126 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-29T04:07:23.171 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-29T04:07:23.211 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-29T04:07:23.241 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:07:23.269 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-29T04:07:23.304 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-29T04:07:23.343 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-29T04:07:23.374 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-29T04:07:23.404 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-29T04:07:23.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-29T04:07:23.775 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-29T04:07:23.923 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-29T04:07:23.965 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-29T04:07:23.999 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-29T04:07:24.089 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-29T04:07:24.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-29T04:07:24.214 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-29T04:07:24.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-29T04:07:24.300 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-29T04:07:24.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-29T04:07:24.446 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-29T04:07:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-29T04:07:25.219 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-29T04:07:25.262 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-29T04:07:25.310 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-29T04:07:25.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-29T04:07:25.438 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-29T04:07:25.615 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-29T04:07:25.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-29T04:07:25.722 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-29T04:07:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-29T04:07:26.466 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-29T04:07:27.685 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-29T04:07:27.761 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-29T04:07:27.792 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-29T04:07:27.823 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-29T04:07:27.850 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-29T04:07:28.163 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-29T04:07:28.189 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:07:28.263 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 124/198 2024-03-29T04:07:28.294 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 125/198 2024-03-29T04:07:28.330 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-29T04:07:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-29T04:07:28.833 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:07:28.904 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 128/198 2024-03-29T04:07:28.927 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 129/198 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-03-29T04:07:29.408 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:29.568 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:07:29.583 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:07:29.583 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:29.583 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:07:29.583 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:31.073 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:07:31.098 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:07:31.098 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:31.099 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:07:31.099 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:07:31.099 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:07:31.099 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:31.235 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:31.275 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:31.323 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:07:31.346 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:07:31.813 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:07:31.814 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:31.814 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:07:31.814 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:07:31.814 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:07:31.814 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:31.818 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 130/198 2024-03-29T04:07:31.861 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:07:31.917 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:07:31.945 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 131/198 2024-03-29T04:07:31.945 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-29T04:07:31.945 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-29T04:07:31.946 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:31.987 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:07:32.366 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:07:32.398 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:07:32.459 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:07:32.489 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:07:33.529 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:07:33.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:07:34.529 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:07:34.557 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:07:34.659 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:07:34.813 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:07:34.843 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:07:34.866 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:34.910 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:07:34.921 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:07:34.976 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:07:35.860 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:07:35.887 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:07:35.910 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:07:35.910 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:35.910 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:07:35.911 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:07:35.911 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:07:35.911 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:35.954 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:07:35.976 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:07:35.977 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:35.977 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:07:35.977 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:36.022 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:07:36.044 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:36.356 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:07:36.377 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:07:36.378 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:36.378 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:07:36.378 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:07:36.378 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:07:36.378 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:40.885 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:07:40.926 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:07:40.957 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:07:40.984 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:07:41.167 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:07:41.301 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:07:41.357 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:07:41.413 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:07:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:07:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:07:41.558 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:07:41.590 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:07:41.619 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:07:41.687 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:07:41.752 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:07:41.784 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:07:42.422 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:42.466 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:42.486 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:43.305 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:07:43.394 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:07:43.435 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:07:43.535 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:07:43.573 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:07:43.598 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:07:43.623 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:07:43.650 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:07:43.650 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:07:43.754 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:07:43.784 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:07:43.784 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:07:43.860 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:07:43.886 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:07:43.920 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:07:43.945 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:07:43.970 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:07:44.005 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:07:44.051 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 132/198 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-03-29T04:07:44.052 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-03-29T04:07:44.053 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-03-29T04:07:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:44.091 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:07:44.116 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:07:44.142 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:07:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:07:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:07:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:44.258 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:07:44.274 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 133/198 2024-03-29T04:07:44.274 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:44.274 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-29T04:07:44.274 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:44.378 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:44.378 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:07:44.470 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:07:44.494 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:07:44.527 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:07:44.554 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:07:44.579 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:07:44.646 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:07:45.770 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 134/198 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-29T04:07:45.796 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:46.020 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 135/198 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-29T04:07:46.043 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:46.479 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:07:46.502 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 136/198 2024-03-29T04:07:46.502 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:46.502 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-29T04:07:46.502 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:07:46.502 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-29T04:07:46.503 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:46.598 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-29T04:07:47.073 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:07:47.104 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 138/198 2024-03-29T04:07:47.136 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 139/198 2024-03-29T04:07:47.198 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 140/198 2024-03-29T04:07:47.227 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:07:48.260 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 141/198 2024-03-29T04:07:48.296 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:07:49.253 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 142/198 2024-03-29T04:07:49.288 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:07:49.389 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 143/198 2024-03-29T04:07:49.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 144/198 2024-03-29T04:07:49.573 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 145/198 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-29T04:07:49.597 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:49.641 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:07:49.652 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 146/198 2024-03-29T04:07:49.714 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 147/198 2024-03-29T04:07:50.172 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:07:50.172 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:07:50.172 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:07:50.172 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:07:50.172 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:07:50.173 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:07:50.174 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:07:50.174 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:07:50.174 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:07:50.174 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:07:50.174 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:07:50.175 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:07:50.176 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:07:50.176 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:07:50.176 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:07:50.176 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:07:50.177 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:07:50.178 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:07:50.178 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:07:50.178 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:07:50.178 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:07:50.178 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:07:50.179 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:07:50.180 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:07:50.180 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:07:50.180 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:07:50.180 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:07:50.180 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:07:50.181 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:07:50.182 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:07:50.182 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:07:50.182 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:07:50.182 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:07:50.182 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:07:50.183 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:07:50.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:07:50.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:07:50.186 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:07:50.186 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:07:50.186 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:07:50.186 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:07:50.186 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:07:50.187 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:07:50.188 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:07:50.189 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:07:50.190 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:07:50.191 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:07:50.192 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:07:50.192 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:07:50.192 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:07:50.192 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:07:50.193 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:07:50.194 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:07:50.195 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:07:50.196 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:07:50.197 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:07:50.198 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:07:50.199 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:07:50.200 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:07:50.201 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:07:50.202 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:07:50.203 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:07:50.203 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:07:50.203 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:07:50.203 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:07:50.203 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:07:50.588 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-29T04:07:50.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 149/198 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-29T04:07:50.642 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:50.685 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:07:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 150/198 2024-03-29T04:07:50.707 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:50.707 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-29T04:07:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:50.752 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-29T04:07:50.775 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:51.098 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 152/198 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-29T04:07:51.121 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:07:53.233 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:07:53.233 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:53.233 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-03-29T04:07:53.233 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.234 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:07:53.235 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.236 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:07:53.237 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.238 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:07:53.239 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:07:53.240 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:07:53.241 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:07:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:07:53.243 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:07:53.244 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:07:53.245 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:07:53.246 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:07:53.247 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:07:53.247 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-03-29T04:07:53.717 DEBUG:teuthology.parallel:result is None 2024-03-29T04:07:55.616 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:07:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 154/198 2024-03-29T04:07:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 155/198 2024-03-29T04:07:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-29T04:07:55.918 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 157/198 2024-03-29T04:07:56.030 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-29T04:07:56.079 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 159/198 2024-03-29T04:07:56.133 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-29T04:07:56.196 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-29T04:07:56.239 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-29T04:07:56.272 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 163/198 2024-03-29T04:07:56.304 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-29T04:07:56.333 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-29T04:07:56.395 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-29T04:07:56.450 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-29T04:07:56.482 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-29T04:07:57.087 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:57.131 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:57.151 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-29T04:07:57.962 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 170/198 2024-03-29T04:07:58.044 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-29T04:07:58.094 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-29T04:07:58.173 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-29T04:07:58.205 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-29T04:07:58.231 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-29T04:07:58.298 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-29T04:07:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-29T04:07:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:07:58.395 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-29T04:07:58.433 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-29T04:07:58.434 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:07:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-29T04:07:58.527 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-29T04:07:58.561 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-29T04:07:58.586 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-29T04:07:58.611 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-29T04:07:58.646 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-29T04:07:58.692 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-29T04:07:58.724 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-29T04:07:58.749 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-29T04:07:58.775 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-29T04:07:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:07:58.854 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-29T04:07:58.861 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:58.861 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:59.053 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-29T04:07:59.053 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:07:59.128 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-29T04:07:59.152 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:07:59.185 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-29T04:07:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-29T04:07:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-29T04:07:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 1/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 3/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 4/198 2024-03-29T04:08:04.707 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2520.g69e2c 5/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 6/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 7/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 8/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 9/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 10/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 11/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 12/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 13/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 14/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_ 15/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 16/198 2024-03-29T04:08:04.708 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 17/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_6 18/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86 19/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 20/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el 21/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9. 22/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_6 23/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 24/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 25/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 26/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 27/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 28/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 29/198 2024-03-29T04:08:04.709 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73. 30/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noa 31/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.n 32/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e 33/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el 34/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 35/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.e 36/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 37/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 38/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 39/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-29T04:08:04.710 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-29T04:08:04.711 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-29T04:08:04.712 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-29T04:08:04.713 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-03-29T04:08:04.714 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-03-29T04:08:04.715 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-29T04:08:04.716 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-29T04:08:04.717 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-29T04:08:04.718 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 151/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 153/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-29T04:08:04.719 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-29T04:08:04.720 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-29T04:08:04.721 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-29T04:08:04.722 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-29T04:08:04.723 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-29T04:08:07.666 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: acl-2.3.1-4.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-29T04:08:07.667 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:07.668 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.669 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.670 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.0.0-2520.g69e2cd73.el9.noarch 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-29T04:08:07.671 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-16.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-29T04:08:07.672 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-29T04:08:07.673 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.14-1.el9.x86_64 2024-03-29T04:08:07.674 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.675 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-29T04:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-29T04:08:07.677 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-29T04:08:07.678 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.679 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-29T04:08:07.680 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.0.0-2520.g69e2cd73.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-3.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-29T04:08:07.681 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-29T04:08:07.682 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-29T04:08:07.682 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:07.682 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-03-29T04:08:08.113 DEBUG:teuthology.parallel:result is None 2024-03-29T04:08:08.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:08:08.348 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:08:08.370 INFO:teuthology.orchestra.run.smithi181.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:08.371 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:08.371 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:08:08.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=69e2cd73c52284dfd5096c66b60c6e6b26a3c447 2024-03-29T04:08:08.600 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T04:08:08.625 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:08.625 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2520.g69e2cd73.el9 2024-03-29T04:08:08.626 INFO:teuthology.task.install:The correct ceph version 19.0.0-2520.g69e2cd73 is installed. 2024-03-29T04:08:08.627 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T04:08:08.628 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:08.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:08:08.658 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:08.658 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T04:08:08.699 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T04:08:08.699 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:08.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:08:08.726 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:08:08.793 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:08.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T04:08:08.823 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T04:08:08.891 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T04:08:08.892 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:08.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:08:08.919 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:08:08.985 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:08.986 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T04:08:09.015 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T04:08:09.086 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T04:08:09.086 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:09.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:08:09.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:08:09.180 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:09.180 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T04:08:09.207 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T04:08:09.275 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T04:08:09.361 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T04:08:09.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2024-03-29T04:08:09.364 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-03-29T04:08:09.392 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T04:08:09.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:08:09.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T04:08:09.459 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T04:08:09.459 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:08:09.500 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T04:08:09.534 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T04:08:09.534 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T04:08:09.534 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T04:08:09.534 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:09.534 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:08:09.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:09.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:09.726 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:09.726 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T04:08:09.741 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:09.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:09.893 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T04:08:09.893 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T04:08:09.893 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629082', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8207, 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBX/vqhpoVv7xQF9OaCd4qKW/a5NFdu1jYL3M9RLEAQtc36m1FhNG7RquWYaxP94P98XjDG4nPnlwf63rzrhEg=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVLz0WdmvAmWumf0aY3sfydbO2fS/7Cp/rwFGjG55ZgtF/39aOqwD8YdyUV7v2Ca0C0PHSlSZxhsByEx8VSW4g='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '69e2cd73c52284dfd5096c66b60c6e6b26a3c447', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_02:01:30', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T04:08:09.894 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:08:09.896 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T04:08:09.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:08:09.952 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T04:08:10.007 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:10.007 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:08:10.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:08:10.032 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-03-29T04:08:10.086 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-03-29 04:07:59.690387231 +0000 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-03-29 04:05:39.988467566 +0000 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-03-29 04:05:39.988467566 +0000 2024-03-29T04:08:10.087 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-03-29 04:05:39.988467566 +0000 2024-03-29T04:08:10.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:08:10.152 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-03-29T04:08:10.152 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-03-29T04:08:10.152 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000260745 s, 2.0 MB/s 2024-03-29T04:08:10.153 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:08:10.209 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-03-29 04:07:59.690387231 +0000 2024-03-29T04:08:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-03-29 04:05:40.331460160 +0000 2024-03-29T04:08:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-03-29 04:05:40.331460160 +0000 2024-03-29T04:08:10.265 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-03-29 04:05:40.331460160 +0000 2024-03-29T04:08:10.265 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:08:10.330 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-03-29T04:08:10.330 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-03-29T04:08:10.330 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000205777 s, 2.5 MB/s 2024-03-29T04:08:10.331 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:08:10.387 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-03-29 04:07:59.690387231 +0000 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-03-29 04:05:40.678452667 +0000 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-03-29 04:05:40.678452667 +0000 2024-03-29T04:08:10.441 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-03-29 04:05:40.678452667 +0000 2024-03-29T04:08:10.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:08:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-03-29T04:08:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-03-29T04:08:10.507 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.00029953 s, 1.7 MB/s 2024-03-29T04:08:10.508 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:08:10.564 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-03-29 04:07:59.691387210 +0000 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-03-29 04:05:41.007445564 +0000 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-03-29 04:05:41.007445564 +0000 2024-03-29T04:08:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-03-29 04:05:41.007445564 +0000 2024-03-29T04:08:10.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:08:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-03-29T04:08:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-03-29T04:08:10.686 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000219291 s, 2.3 MB/s 2024-03-29T04:08:10.687 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:08:10.743 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:10.743 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:10.743 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T04:08:10.760 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T04:08:10.760 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-03-29 04:07:45.020650872 +0000 2024-03-29T04:08:10.816 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-03-29 04:05:29.925575140 +0000 2024-03-29T04:08:10.817 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-03-29 04:05:29.925575140 +0000 2024-03-29T04:08:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-03-29 04:05:29.925575140 +0000 2024-03-29T04:08:10.817 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T04:08:10.884 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-03-29T04:08:10.885 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-03-29T04:08:10.885 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000185322 s, 2.8 MB/s 2024-03-29T04:08:10.885 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T04:08:10.942 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-03-29T04:08:10.996 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-03-29 04:07:45.021650856 +0000 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-03-29 04:05:30.261569624 +0000 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-03-29 04:05:30.261569624 +0000 2024-03-29T04:08:10.997 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-03-29 04:05:30.261569624 +0000 2024-03-29T04:08:10.997 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T04:08:11.062 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-03-29T04:08:11.062 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-03-29T04:08:11.062 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000183787 s, 2.8 MB/s 2024-03-29T04:08:11.063 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T04:08:11.119 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-03-29T04:08:11.174 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T04:08:11.174 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:11.174 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-29T04:08:11.174 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:11.175 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:11.175 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-03-29 04:07:45.021650856 +0000 2024-03-29T04:08:11.175 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-03-29 04:05:30.598564093 +0000 2024-03-29T04:08:11.175 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-03-29 04:05:30.598564093 +0000 2024-03-29T04:08:11.175 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-03-29 04:05:30.598564093 +0000 2024-03-29T04:08:11.175 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T04:08:11.240 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-03-29T04:08:11.240 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-03-29T04:08:11.240 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000209576 s, 2.4 MB/s 2024-03-29T04:08:11.241 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T04:08:11.296 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-03-29T04:08:11.351 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T04:08:11.351 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T04:08:11.351 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-29T04:08:11.351 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T04:08:11.352 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T04:08:11.352 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-03-29 04:07:45.021650856 +0000 2024-03-29T04:08:11.352 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-03-29 04:05:30.939558496 +0000 2024-03-29T04:08:11.352 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-03-29 04:05:30.939558496 +0000 2024-03-29T04:08:11.352 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-03-29 04:05:30.939558496 +0000 2024-03-29T04:08:11.352 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T04:08:11.417 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-03-29T04:08:11.417 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-03-29T04:08:11.417 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.00018338 s, 2.8 MB/s 2024-03-29T04:08:11.418 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T04:08:11.474 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:11.475 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:08:11.475 INFO:tasks.ceph:Generating config... 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] pid file = 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T04:08:11.476 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T04:08:11.476 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-29T04:08:11.476 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T04:08:11.476 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T04:08:11.477 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T04:08:11.478 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-29T04:08:11.478 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T04:08:11.478 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T04:08:11.479 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T04:08:11.479 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T04:08:11.479 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T04:08:11.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T04:08:11.526 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T04:08:11.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T04:08:11.573 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:08:11.639 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.181'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-03-29T04:08:11.640 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.181,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T04:08:11.640 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:11.641 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:08:11.694 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.181 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: generated fsid fab9a006-093b-45b9-834f-2720bd7aa224 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:setting min_mon_release = quincy 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 0 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:fsid fab9a006-093b-45b9-834f-2720bd7aa224 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:last_changed 2024-03-29T04:08:11.800489+0000 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:created 2024-03-29T04:08:11.800489+0000 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:min_mon_release 17 (quincy) 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:election_strategy: 1 2024-03-29T04:08:11.779 INFO:teuthology.orchestra.run.smithi181.stdout:0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2024-03-29T04:08:11.780 INFO:teuthology.orchestra.run.smithi181.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-03-29T04:08:11.780 INFO:teuthology.orchestra.run.smithi181.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-03-29T04:08:11.780 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T04:08:11.781 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T04:08:11.795 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fab9a006-093b-45b9-834f-2720bd7aa224... 2024-03-29T04:08:11.796 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:08:11.838 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T04:08:11.885 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2024-03-29T04:08:11.885 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2024-03-29T04:08:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:08:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2024-03-29T04:08:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.181,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on skipped message = False 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = fab9a006-093b-45b9-834f-2720bd7aa224 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug trim objects = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 20 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 20 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 10 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:08:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2024-03-29T04:08:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: debug finisher = 20 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:08:11.893 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon down mkfs grace = 300 2024-03-29T04:08:11.894 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2024-03-29T04:08:11.895 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds balancer = 20 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2024-03-29T04:08:11.896 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi181.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-03-29T04:08:11.897 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-03-29T04:08:11.898 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T04:08:11.899 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.181,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-03-29T04:08:11.900 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = fab9a006-093b-45b9-834f-2720bd7aa224 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-03-29T04:08:11.901 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-03-29T04:08:11.902 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-03-29T04:08:11.903 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug finisher = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-03-29T04:08:11.904 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.905 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-03-29T04:08:11.906 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: mds_bal_replicate_threshold = 1 2024-03-29T04:08:11.907 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-03-29T04:08:11.908 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-03-29T04:08:11.908 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-03-29T04:08:11.909 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T04:08:11.909 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T04:08:12.008 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T04:08:12.008 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:12.008 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T04:08:12.024 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:12.024 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T04:08:12.079 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:08:12.079 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:12.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:08:12.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:08:12.158 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:12.159 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:12.213 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:08:12.213 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:12.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T04:08:12.213 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T04:08:12.253 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:12.253 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:12.307 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T04:08:12.307 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T04:08:12.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:08:12.362 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T04:08:12.366 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:08:12.440 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T04:08:12.442 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T04:08:12.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:08:12.497 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T04:08:12.500 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T04:08:12.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:08:12.621 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T04:08:12.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T04:08:12.652 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:08:12.707 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T04:08:12.709 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T04:08:12.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:08:12.830 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T04:08:12.833 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T04:08:12.859 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T04:08:12.860 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T04:08:12.902 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T04:08:12.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T04:08:13.001 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T04:08:13.016 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T04:08:13.057 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T04:08:13.073 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T04:08:13.154 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T04:08:13.169 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T04:08:13.170 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T04:08:13.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T04:08:13.198 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:13.198 INFO:tasks.ceph:role: osd.0 2024-03-29T04:08:13.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:08:13.198 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:13.273 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:13.274 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:13.274 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:13.274 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:13.781 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-03-29T04:08:13.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-03-29T04:08:13.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T04:08:13.935 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T04:08:13.973 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T04:08:14.043 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:14.043 INFO:tasks.ceph:role: osd.1 2024-03-29T04:08:14.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:08:14.043 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:08:14.118 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:14.118 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:14.118 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:14.118 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:14.119 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:14.579 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-03-29T04:08:14.582 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-03-29T04:08:14.582 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T04:08:14.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T04:08:14.711 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T04:08:14.779 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:14.779 INFO:tasks.ceph:role: osd.2 2024-03-29T04:08:14.779 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:08:14.779 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:14.853 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:14.854 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:14.854 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:14.854 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:15.340 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-03-29T04:08:15.342 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-03-29T04:08:15.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T04:08:15.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T04:08:15.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T04:08:15.596 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:15.596 INFO:tasks.ceph:role: osd.3 2024-03-29T04:08:15.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:08:15.596 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:15.671 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:16.176 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-03-29T04:08:16.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-03-29T04:08:16.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T04:08:16.322 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T04:08:16.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:16.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T04:08:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T04:08:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:08:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:08:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T04:08:16.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:16.473+0000 7f8d3ba1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T04:08:19.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T04:08:19.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.448+0000 7f757ec0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.448+0000 7f757ec0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.448+0000 7f757ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.448+0000 7f757ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.448+0000 7f757ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T04:08:19.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:19.449+0000 7f757ec0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T04:08:22.277 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T04:08:22.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.419+0000 7f1550211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T04:08:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.419+0000 7f1550211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T04:08:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.419+0000 7f1550211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:08:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.419+0000 7f1550211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:08:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.419+0000 7f1550211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T04:08:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:22.420+0000 7f1550211640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T04:08:25.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T04:08:25.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:25.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.393+0000 7fb52580d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T04:08:25.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.394+0000 7fb52580d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T04:08:25.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.394+0000 7fb52580d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:08:25.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.394+0000 7fb52580d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:08:25.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.394+0000 7fb52580d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T04:08:25.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:25.394+0000 7fb52580d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T04:08:28.220 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T04:08:28.251 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T04:08:28.283 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:28.283 INFO:tasks.ceph:role: osd.4 2024-03-29T04:08:28.283 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:08:28.283 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T04:08:28.361 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:28.362 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:28.872 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-03-29T04:08:28.874 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-03-29T04:08:28.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T04:08:28.993 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T04:08:29.030 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T04:08:29.099 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:29.100 INFO:tasks.ceph:role: osd.5 2024-03-29T04:08:29.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:08:29.100 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T04:08:29.176 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:29.176 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:29.176 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:29.176 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:29.177 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:29.630 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-03-29T04:08:29.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-03-29T04:08:29.632 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T04:08:29.723 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T04:08:29.754 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T04:08:29.823 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:29.823 INFO:tasks.ceph:role: osd.6 2024-03-29T04:08:29.823 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:08:29.823 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T04:08:29.897 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:29.898 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:30.387 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-03-29T04:08:30.389 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-03-29T04:08:30.389 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T04:08:30.531 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T04:08:30.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T04:08:30.629 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T04:08:30.629 INFO:tasks.ceph:role: osd.7 2024-03-29T04:08:30.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:08:30.629 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T04:08:30.704 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T04:08:30.704 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T04:08:30.704 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T04:08:30.704 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-29T04:08:30.704 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T04:08:30.705 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-03-29T04:08:30.705 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T04:08:30.705 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-29T04:08:30.705 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T04:08:30.705 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T04:08:31.208 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-03-29T04:08:31.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-03-29T04:08:31.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T04:08:31.340 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T04:08:31.377 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:31.474 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.494+0000 7fd6ce613640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T04:08:31.474 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.494+0000 7fd6ce613640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T04:08:31.474 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.494+0000 7fd6ce613640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:08:31.474 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.494+0000 7fd6ce613640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:08:31.474 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.494+0000 7fd6ce613640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T04:08:31.475 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:31.496+0000 7fd6ce613640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T04:08:34.331 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T04:08:34.362 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:34.453 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.473+0000 7fbf803f6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T04:08:34.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.473+0000 7fbf803f6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T04:08:34.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.473+0000 7fbf803f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:08:34.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.474+0000 7fbf803f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:08:34.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.474+0000 7fbf803f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T04:08:34.454 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:34.474+0000 7fbf803f6640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T04:08:37.308 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T04:08:37.339 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.450+0000 7f2936302640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.450+0000 7f2936302640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.451+0000 7f2936302640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.451+0000 7f2936302640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.451+0000 7f2936302640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T04:08:37.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:37.451+0000 7f2936302640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T04:08:39.895 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:39.899 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:08:40.283 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T04:08:40.314 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T04:08:40.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:40.426+0000 7f22722a4640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T04:08:43.268 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T04:08:43.300 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T04:08:43.300 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.301 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T04:08:43.332 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T04:08:43.402 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T04:08:43.472 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T04:08:43.541 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.542 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T04:08:43.611 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.611 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T04:08:43.680 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:43.680 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T04:08:43.747 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:43.747 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T04:08:43.779 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:43.779 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T04:08:43.849 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:43.849 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T04:08:43.919 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:43.920 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T04:08:43.988 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:43.988 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T04:08:44.057 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:44.057 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T04:08:44.125 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:44.125 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T04:08:44.193 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:44.193 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T04:08:44.209 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:44.210 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T04:08:44.265 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:44.265 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T04:08:44.280 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:44.280 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T04:08:44.335 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T04:08:44.335 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:08:44.338 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZm7VLeFhAAxF0G6FHc+xEuTPHFnIx85A== 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZm+TbyHhAAYS5vSs4/+lQVVlgLg3zhHw== 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZmu+FXJhAARrpjcV/v0swLv8eCgzyfMA== 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCwPgZmuIVAHBAAaGjP9Mlwc0CDVLVSXbAkwg== 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCzPgZmmX/GGhAAiihsU2WYAp46yWDHHzBfMA== 2024-03-29T04:08:44.379 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC2PgZmxVUOGRAAE2idq11w4BUXWG60f96J4A== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC5PgZmbn2HFxAAIjWELGbJT6yyL2rKOckWcA== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZmjT+EGxAAv1PsM4wkpglDq1aHTMHodQ== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZmEsdzKxAAW5MeNb4ibn6ssrofWq+OIA== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZmGlPPMhAAJAvKfZWz8U9XtO4+O8535Q== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQC/PgZmKV2GHRAAycPnjdnl/wtlL+R6zCX51Q== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDCPgZmAY5PHBAAOG9GOPUe/fVQi6FfZg3tPQ== 2024-03-29T04:08:44.380 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDFPgZmRdvwGhAAxpNEqLXzPuG9d6PVNBhJiw== 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDIPgZmnQN5GRAA3k64GtOezCHgN2msx/+jEg== 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZm/aIPNxAAtuCXeNhwViJIHObDAGDzrw== 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCtPgZmPYBWARAATTtpn5FUlfeGNmSnUOKdVg== 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCtPgZm0P69BBAA6BlHVG2jLo5NyRVPct4M8Q== 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2024-03-29T04:08:44.381 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCtPgZm1Zt/ChAArZhRTQqCS4S1dBOBh+imWw== 2024-03-29T04:08:44.401 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-03-29T04:08:44.401 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZm7VLeFhAAxF0G6FHc+xEuTPHFnIx85A== 2024-03-29T04:08:44.401 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZm+TbyHhAAYS5vSs4/+lQVVlgLg3zhHw== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZmu+FXJhAARrpjcV/v0swLv8eCgzyfMA== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCwPgZmuIVAHBAAaGjP9Mlwc0CDVLVSXbAkwg== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCzPgZmmX/GGhAAiihsU2WYAp46yWDHHzBfMA== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQC2PgZmxVUOGRAAE2idq11w4BUXWG60f96J4A== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQC5PgZmbn2HFxAAIjWELGbJT6yyL2rKOckWcA== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZmjT+EGxAAv1PsM4wkpglDq1aHTMHodQ== 2024-03-29T04:08:44.402 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZmEsdzKxAAW5MeNb4ibn6ssrofWq+OIA== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZmGlPPMhAAJAvKfZWz8U9XtO4+O8535Q== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQC/PgZmKV2GHRAAycPnjdnl/wtlL+R6zCX51Q== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDCPgZmAY5PHBAAOG9GOPUe/fVQi6FfZg3tPQ== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDFPgZmRdvwGhAAxpNEqLXzPuG9d6PVNBhJiw== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDIPgZmnQN5GRAA3k64GtOezCHgN2msx/+jEg== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCsPgZm/aIPNxAAtuCXeNhwViJIHObDAGDzrw== 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2024-03-29T04:08:44.403 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCtPgZmPYBWARAATTtpn5FUlfeGNmSnUOKdVg== 2024-03-29T04:08:44.404 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-03-29T04:08:44.404 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCtPgZm0P69BBAA6BlHVG2jLo5NyRVPct4M8Q== 2024-03-29T04:08:44.404 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2024-03-29T04:08:44.404 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCtPgZm1Zt/ChAArZhRTQqCS4S1dBOBh+imWw== 2024-03-29T04:08:44.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:08:44.425 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:08:44.498 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.501 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.559 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.618 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.670 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.672 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.727 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.778 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.780 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:44.832 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:08:44.836 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T04:08:44.887 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.890 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:44.948 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T04:08:45.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.005 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.056 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.110 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.112 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.166 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T04:08:45.218 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.220 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.277 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.334 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.390 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T04:08:45.445 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T04:08:45.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T04:08:45.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:08:45.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T04:08:45.596 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T04:08:45.623 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:08:45.780 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T04:08:45.807 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T04:08:45.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T04:08:45.971 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T04:08:45.997 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:46.000 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T04:08:46.052 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T04:08:46.052 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T04:08:46.052 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T04:08:46.057 INFO:tasks.ceph.mon.a:Started 2024-03-29T04:08:46.057 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T04:08:46.057 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T04:08:46.094 INFO:tasks.ceph.mon.b:Started 2024-03-29T04:08:46.094 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T04:08:46.094 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T04:08:46.096 INFO:tasks.ceph.mon.c:Started 2024-03-29T04:08:46.096 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T04:08:46.096 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T04:08:46.096 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T04:08:46.098 INFO:tasks.ceph.mgr.y:Started 2024-03-29T04:08:46.099 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T04:08:46.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T04:08:46.101 INFO:tasks.ceph.mgr.x:Started 2024-03-29T04:08:46.101 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T04:08:46.101 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T04:08:46.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.224+0000 7fb2eb5f2640 1 Processor -- start 2024-03-29T04:08:46.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.224+0000 7fb2eb5f2640 1 -- start start 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2eb5f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2eb5f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2eb5f2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e40d2670 con 0x7fb2e40b4300 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2eb5f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e40cc8e0 con 0x7fb2e40b3f20 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2eb5f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e40ccbc0 con 0x7fb2e40d1ca0 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57896/0 (socket says 172.21.15.181:57896) 2024-03-29T04:08:46.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 learned_addr learned my addr 172.21.15.181:0/1257645744 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.225+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.226+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.226+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.226+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.226+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:08:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.426+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.426+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.426+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.426+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.426+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:08:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.427+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.427+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.427+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.427+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.427+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:08:46.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.827+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.827+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:46.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.827+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.827+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.827+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:08:46.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.828+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:46.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.828+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:46.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.828+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:46.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.828+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:46.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:46.828+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:08:47.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.628+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.628+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:47.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.628+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:47.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.628+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:47.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.628+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:08:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.629+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.629+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.629+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.629+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:47.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:47.629+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 msgr2=0x7fb2e40d2130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.187:3300/0 conn(0x7fb2e40d1ca0 0x7fb2e40d2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 -- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 msgr2=0x7fb2e415d2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 --2- 172.21.15.181:0/1257645744 >> v2:172.21.15.181:3300/0 conn(0x7fb2e40b3f20 0x7fb2e415d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 -- 172.21.15.181:0/1257645744 >> v1:172.21.15.187:6789/0 conn(0x7fb2e40b4300 legacy=0x7fb2e40ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 -- 172.21.15.181:0/1257645744 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e415cac0 con 0x7fb2c8006150 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2daffd640 1 -- 172.21.15.181:0/1257645744 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e40cc8e0 con 0x7fb2c80036b0 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:49.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:49.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:49.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:49.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.226+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T04:08:49.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.427+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:49.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.427+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:49.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.427+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:49.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.427+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:49.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.427+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T04:08:49.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.828+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:49.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.828+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:49.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.828+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:49.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.828+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:49.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:49.828+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T04:08:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:50.629+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:50.629+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T04:08:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:50.629+0000 7fb2e938e640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T04:08:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:50.629+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T04:08:50.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:50.629+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T04:08:51.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.394+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384913672 0 0) 0x7fb2e415cac0 con 0x7fb2c8006150 2024-03-29T04:08:51.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.394+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2d0003520 con 0x7fb2c8006150 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2470475199 0 0) 0x7fb2d0003520 con 0x7fb2c8006150 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 msgr2=0x7fb2c8005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c80036b0 0x7fb2c8005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c800afb0 con 0x7fb2c8006150 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fb2d40016f0 con 0x7fb2c8006150 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 >> v1:172.21.15.181:6789/0 conn(0x7fb2c8006150 legacy=0x7fb2c8008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/1257645744 reap_dead start 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d00039b0 0x7fb2d0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 0x7fb2d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 0x7fb2d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40cc8e0 con 0x7fb2d0004800 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d000c1e0 con 0x7fb2d00039b0 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d000c3e0 con 0x7fb2d0005290 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.395+0000 7fb2e8b8d640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 0x7fb2d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e938e640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d00039b0 0x7fb2d0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 0x7fb2d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e8b8d640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 msgr2=0x7fb2d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e8b8d640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 0x7fb2d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e8b8d640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d00039b0 msgr2=0x7fb2d0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e8b8d640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d00039b0 0x7fb2d0003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e8b8d640 1 -- 172.21.15.181:0/1257645744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d0003520 con 0x7fb2d0004800 2024-03-29T04:08:51.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.396+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 0x7fb2d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:51.378 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.397+0000 7fb2e8b8d640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 0x7fb2d0004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c110 tx=0x7fb2d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629c786112d61c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.397+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4007c40 con 0x7fb2d0004800 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.397+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d4030070 con 0x7fb2d0004800 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.397+0000 7fb2dbfff640 1 -- 172.21.15.181:0/1257645744 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d402cc50 con 0x7fb2d0004800 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 msgr2=0x7fb2d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 0x7fb2d0004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c110 tx=0x7fb2d0005830 comp rx=0 tx=0).stop 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/1257645744 shutdown_connections 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d0005290 0x7fb2d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0004800 0x7fb2d0004c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/1257645744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d00039b0 0x7fb2d0003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/1257645744 >> 172.21.15.181:0/1257645744 conn(0x7fb2e40672c0 msgr2=0x7fb2e40676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/1257645744 shutdown_connections 2024-03-29T04:08:51.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.398+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/1257645744 wait complete. 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.400+0000 7fb2eb5f2640 1 Processor -- start 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.400+0000 7fb2eb5f2640 1 -- start start 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 0x7fb2e40c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d0004800 0x7fb2e40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40d22b0 con 0x7fb2d0005290 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e40d2130 con 0x7fb2d0004800 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2eb5f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e40d2430 con 0x7fb2d00039b0 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e938e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 0x7fb2e40c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59844/0 (socket says 172.21.15.181:59844) 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/195682215 learned_addr learned my addr 172.21.15.181:0/195682215 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e8b8d640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d0004800 0x7fb2e40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e938e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 0x7fb2e40c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:52154/0 (socket says 172.21.15.181:52154) 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 msgr2=0x7fb2e40c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 0x7fb2e40c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d0004800 msgr2=0x7fb2e40c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d0004800 0x7fb2e40c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 -- 172.21.15.181:0/195682215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e40caa80 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.401+0000 7fb2e9b8f640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009610 tx=0x7fb2dc009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc023070 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc01e3f0 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc01e7c0 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e40ca090 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e40cb000 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2dc01e960 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.402+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2dc027da0 con 0x7fb2d0005290 2024-03-29T04:08:51.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.404+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e415cac0 con 0x7fb2d0005290 2024-03-29T04:08:51.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.407+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb2e415cac0 con 0x7fb2d0005290 2024-03-29T04:08:51.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.472+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb2e40d1a80 con 0x7fb2d0005290 2024-03-29T04:08:51.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.476+0000 7fb2d9ffb640 1 -- 172.21.15.181:0/195682215 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb2dc02c020 con 0x7fb2d0005290 2024-03-29T04:08:51.455 INFO:teuthology.orchestra.run.smithi181.stderr:adjusted tunables profile to default 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 msgr2=0x7fb2e41f0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009610 tx=0x7fb2dc009d70 comp rx=0 tx=0).stop 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 shutdown_connections 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2d0005290 0x7fb2e41f0110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d0004800 0x7fb2e40c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 --2- 172.21.15.181:0/195682215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2d00039b0 0x7fb2e40c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 >> 172.21.15.181:0/195682215 conn(0x7fb2e40672c0 msgr2=0x7fb2e40676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 shutdown_connections 2024-03-29T04:08:51.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:51.477+0000 7fb2eb5f2640 1 -- 172.21.15.181:0/195682215 wait complete. 2024-03-29T04:08:51.468 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T04:08:51.468 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T04:08:51.469 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:51.469 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T04:08:51.496 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:51.496 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T04:08:51.563 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:51.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T04:08:51.630 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:08:51.630 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T04:08:51.697 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:51.697 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T04:08:51.725 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:51.725 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T04:08:51.791 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:51.791 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T04:08:51.846 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:51.866+0000 7f9305e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:08:51.858 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:08:51.858 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T04:08:51.883 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:51.904+0000 7ff5fbe4b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T04:08:51.902 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:51.923+0000 7f9305e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:08:51.925 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 642f3f8a-0d64-48be-9111-570fa2cfcb5c 0 2024-03-29T04:08:51.936 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:51.957+0000 7ff5fbe4b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T04:08:52.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.086+0000 7f8a3704e640 1 Processor -- start 2024-03-29T04:08:52.066 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.086+0000 7f8a3704e640 1 -- start start 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a3704e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 0x7f8a3015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a3704e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 0x7f8a300d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a3704e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a300d2670 con 0x7f8a300b4300 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a3704e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8a300cc8e0 con 0x7f8a300d1ca0 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a3704e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a300ccbc0 con 0x7f8a300b3f20 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a34dea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 0x7f8a3015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a2ffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f8a300b4300 0x7f8a300ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:36786/0 (socket says 172.21.15.187:36786) 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a2ffff640 1 -- 172.21.15.187:0/1771258608 learned_addr learned my addr 172.21.15.187:0/1771258608 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a355eb640 1 --2- 172.21.15.187:0/1771258608 >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 0x7f8a300d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a2f7fe640 1 -- 172.21.15.187:0/1771258608 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596257611 0 0) 0x7f8a1c001110 con 0x7f8a300b4300 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a2f7fe640 1 -- 172.21.15.187:0/1771258608 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a20003520 con 0x7f8a300b4300 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a34dea640 1 -- 172.21.15.187:0/1771258608 >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 msgr2=0x7f8a300d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a34dea640 1 --2- 172.21.15.187:0/1771258608 >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 0x7f8a300d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a34dea640 1 -- 172.21.15.187:0/1771258608 >> v1:172.21.15.187:6789/0 conn(0x7f8a300b4300 legacy=0x7f8a300ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a34dea640 1 -- 172.21.15.187:0/1771258608 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3015cac0 con 0x7f8a300b3f20 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.087+0000 7f8a355eb640 1 --2- 172.21.15.187:0/1771258608 >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 0x7f8a300d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.088+0000 7f8a34dea640 1 --2- 172.21.15.187:0/1771258608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 0x7f8a3015d2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a240097a0 tx=0x7f8a2402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262040858657a17f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.088+0000 7f8a2f7fe640 1 -- 172.21.15.187:0/1771258608 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240315a0 con 0x7f8a300b3f20 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.088+0000 7f8a2f7fe640 1 -- 172.21.15.187:0/1771258608 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a24037070 con 0x7f8a300b3f20 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.088+0000 7f8a2f7fe640 1 -- 172.21.15.187:0/1771258608 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24031940 con 0x7f8a300b3f20 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 -- 172.21.15.187:0/1771258608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 msgr2=0x7f8a3015d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 --2- 172.21.15.187:0/1771258608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 0x7f8a3015d2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a240097a0 tx=0x7f8a2402b840 comp rx=0 tx=0).stop 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 -- 172.21.15.187:0/1771258608 shutdown_connections 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 --2- 172.21.15.187:0/1771258608 >> v2:172.21.15.181:3300/0 conn(0x7f8a300d1ca0 0x7f8a300d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 --2- 172.21.15.187:0/1771258608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300b3f20 0x7f8a3015d2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 -- 172.21.15.187:0/1771258608 >> 172.21.15.187:0/1771258608 conn(0x7f8a300672c0 msgr2=0x7f8a300676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 -- 172.21.15.187:0/1771258608 shutdown_connections 2024-03-29T04:08:52.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.089+0000 7f8a3704e640 1 -- 172.21.15.187:0/1771258608 wait complete. 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.090+0000 7f8a3704e640 1 Processor -- start 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.090+0000 7f8a3704e640 1 -- start start 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a300b3f20 0x7f8a301e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300d1ca0 0x7f8a3014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300cd020 con 0x7f8a300b3f20 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a300ccea0 con 0x7f8a300b4300 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a3015d800 con 0x7f8a300d1ca0 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a355eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300d1ca0 0x7f8a3014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.073 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49234/0 (socket says 172.21.15.187:49234) 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 -- 172.21.15.187:0/3553943978 learned_addr learned my addr 172.21.15.187:0/3553943978 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a34dea640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a300b3f20 0x7f8a301e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 -- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300d1ca0 msgr2=0x7f8a3014cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300d1ca0 0x7f8a3014cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 -- 172.21.15.187:0/3553943978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a300b3f20 msgr2=0x7f8a301e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a300b3f20 0x7f8a301e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 -- 172.21.15.187:0/3553943978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a300c75e0 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2ffff640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c760 tx=0x7f8a1c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c017070 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c005ad0 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a300c78c0 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c007e20 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.091+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a300c7e80 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.092+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a1c01a070 con 0x7f8a300b4300 2024-03-29T04:08:52.074 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.092+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f8a1c01c440 con 0x7f8a300b4300 2024-03-29T04:08:52.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.092+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a30075890 con 0x7f8a300b4300 2024-03-29T04:08:52.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.094+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8a1c021020 con 0x7f8a300b4300 2024-03-29T04:08:52.144 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.164+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "642f3f8a-0d64-48be-9111-570fa2cfcb5c", "id": 0} v 0) -- 0x7f8a300cc5b0 con 0x7f8a300b4300 2024-03-29T04:08:52.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.168+0000 7f8a2d7fa640 1 -- 172.21.15.187:0/3553943978 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "642f3f8a-0d64-48be-9111-570fa2cfcb5c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f8a300d2670 con 0x7f8a300b4300 2024-03-29T04:08:52.148 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 msgr2=0x7f8a301e5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c760 tx=0x7f8a1c00cc20 comp rx=0 tx=0).stop 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 shutdown_connections 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a300d1ca0 0x7f8a3014cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a300b4300 0x7f8a301e5cf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 --2- 172.21.15.187:0/3553943978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a300b3f20 0x7f8a301e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 >> 172.21.15.187:0/3553943978 conn(0x7f8a300672c0 msgr2=0x7f8a300676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 shutdown_connections 2024-03-29T04:08:52.150 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.169+0000 7f8a3704e640 1 -- 172.21.15.187:0/3553943978 wait complete. 2024-03-29T04:08:52.159 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 71ca0663-03e6-45f6-b7fd-85c2eb9445ef 1 2024-03-29T04:08:52.259 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.279+0000 7fe02164a640 1 Processor -- start 2024-03-29T04:08:52.259 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.279+0000 7fe02164a640 1 -- start start 2024-03-29T04:08:52.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe02164a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe02164a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe01c15ec60 0x7fe01c0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe02164a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01c0b49f0 con 0x7fe01c0cf890 2024-03-29T04:08:52.261 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe02164a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c0b47c0 con 0x7fe01c15ec60 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe02164a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c070720 con 0x7fe01c154fb0 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe01c15ec60 0x7fe01c0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:41262/0 (socket says 172.21.15.187:41262) 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe01c15ec60 0x7fe01c0b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49248/0 (socket says 172.21.15.187:49248) 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01affd640 1 -- 172.21.15.187:0/1307554748 learned_addr learned my addr 172.21.15.187:0/1307554748 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01b7fe640 1 -- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 msgr2=0x7fe01c0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.280+0000 7fe01b7fe640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe01b7fe640 1 -- 172.21.15.187:0/1307554748 >> v1:172.21.15.181:6789/0 conn(0x7fe01c0cf890 legacy=0x7fe01c15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe01b7fe640 1 -- 172.21.15.187:0/1307554748 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0cead0 con 0x7fe01c15ec60 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe01affd640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe01b7fe640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c15ec60 0x7fe01c0b4280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe014009dc0 tx=0x7fe01402be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f9c10711fb197d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe019ffb640 1 -- 172.21.15.187:0/1307554748 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014039070 con 0x7fe01c15ec60 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe019ffb640 1 -- 172.21.15.187:0/1307554748 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01402cbb0 con 0x7fe01c15ec60 2024-03-29T04:08:52.262 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.281+0000 7fe019ffb640 1 -- 172.21.15.187:0/1307554748 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014034440 con 0x7fe01c15ec60 2024-03-29T04:08:52.265 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 -- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c15ec60 msgr2=0x7fe01c0b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.265 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c15ec60 0x7fe01c0b4280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe014009dc0 tx=0x7fe01402be00 comp rx=0 tx=0).stop 2024-03-29T04:08:52.265 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 -- 172.21.15.187:0/1307554748 shutdown_connections 2024-03-29T04:08:52.265 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c15ec60 0x7fe01c0b4280 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 --2- 172.21.15.187:0/1307554748 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 -- 172.21.15.187:0/1307554748 >> 172.21.15.187:0/1307554748 conn(0x7fe01c0cd310 msgr2=0x7fe01c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 -- 172.21.15.187:0/1307554748 shutdown_connections 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.282+0000 7fe02164a640 1 -- 172.21.15.187:0/1307554748 wait complete. 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.283+0000 7fe02164a640 1 Processor -- start 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.283+0000 7fe02164a640 1 -- start start 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 0x7fe01c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0cf370 con 0x7fe01c15ec60 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c0cf090 con 0x7fe01c0cf890 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe02164a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c155390 con 0x7fe01c154fb0 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.266 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01a7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49256/0 (socket says 172.21.15.187:49256) 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 -- 172.21.15.187:0/133616362 learned_addr learned my addr 172.21.15.187:0/133616362 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01b7fe640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 0x7fe01c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 -- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 msgr2=0x7fe01c153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 -- 172.21.15.187:0/133616362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 msgr2=0x7fe01c156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 0x7fe01c156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01affd640 1 -- 172.21.15.187:0/133616362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1547c0 con 0x7fe01c0cf890 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01a7fc640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.284+0000 7fe01b7fe640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 0x7fe01c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe01affd640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e980 tx=0x7fe01000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01000b040 con 0x7fe01c0cf890 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe01000cd00 con 0x7fe01c0cf890 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c151750 con 0x7fe01c0cf890 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010004230 con 0x7fe01c0cf890 2024-03-29T04:08:52.267 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c154460 con 0x7fe01c0cf890 2024-03-29T04:08:52.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe0100039a0 con 0x7fe01c0cf890 2024-03-29T04:08:52.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.285+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fe01001ad60 con 0x7fe01c0cf890 2024-03-29T04:08:52.268 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.286+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c075890 con 0x7fe01c0cf890 2024-03-29T04:08:52.269 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.289+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe010016020 con 0x7fe01c0cf890 2024-03-29T04:08:52.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.362+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "71ca0663-03e6-45f6-b7fd-85c2eb9445ef", "id": 1} v 0) -- 0x7fe01c0cd0f0 con 0x7fe01c0cf890 2024-03-29T04:08:52.347 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.368+0000 7fe0037fe640 1 -- 172.21.15.187:0/133616362 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "71ca0663-03e6-45f6-b7fd-85c2eb9445ef", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fe010003e20 con 0x7fe01c0cf890 2024-03-29T04:08:52.348 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 msgr2=0x7fe01c14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e980 tx=0x7fe01000ee40 comp rx=0 tx=0).stop 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 shutdown_connections 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe01c15ec60 0x7fe01c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe01c154fb0 0x7fe01c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.349 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 --2- 172.21.15.187:0/133616362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe01c0cf890 0x7fe01c14c110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 >> 172.21.15.187:0/133616362 conn(0x7fe01c0cd310 msgr2=0x7fe01c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 shutdown_connections 2024-03-29T04:08:52.350 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.369+0000 7fe02164a640 1 -- 172.21.15.187:0/133616362 wait complete. 2024-03-29T04:08:52.359 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new d06b03de-42fc-4295-a039-9514a94cb7b2 2 2024-03-29T04:08:52.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.481+0000 7f19709e2640 1 Processor -- start 2024-03-29T04:08:52.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.481+0000 7f19709e2640 1 -- start start 2024-03-29T04:08:52.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f19709e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f196c155b10 0x7f196c0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f19709e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 0x7f196c0b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f19709e2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c0b5ff0 con 0x7f196c0d03c0 2024-03-29T04:08:52.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f19709e2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f196c155ef0 con 0x7f196c15fae0 2024-03-29T04:08:52.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f19709e2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f196c0b41b0 con 0x7f196c155b10 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f196a59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f196c155b10 0x7f196c0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f196a59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f196c155b10 0x7f196c0b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49262/0 (socket says 172.21.15.187:49262) 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f196a59c640 1 -- 172.21.15.187:0/99161187 learned_addr learned my addr 172.21.15.187:0/99161187 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.482+0000 7f1963fff640 1 --2- 172.21.15.187:0/99161187 >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 0x7f196c0b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f196a59c640 1 -- 172.21.15.187:0/99161187 >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 msgr2=0x7f196c0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f196a59c640 1 --2- 172.21.15.187:0/99161187 >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 0x7f196c0b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f196a59c640 1 -- 172.21.15.187:0/99161187 >> v1:172.21.15.181:6789/0 conn(0x7f196c0d03c0 legacy=0x7f196c15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f196a59c640 1 -- 172.21.15.187:0/99161187 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0b4490 con 0x7f196c155b10 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f1963fff640 1 --2- 172.21.15.187:0/99161187 >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 0x7f196c0b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f196a59c640 1 --2- 172.21.15.187:0/99161187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c0b4c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f196c0b7380 tx=0x7f195c02dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d7663fbd97bf9d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.483+0000 7f1969d9b640 1 -- 172.21.15.187:0/99161187 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c03b070 con 0x7f196c155b10 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f1969d9b640 1 -- 172.21.15.187:0/99161187 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c02e960 con 0x7f196c155b10 2024-03-29T04:08:52.464 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f1969d9b640 1 -- 172.21.15.187:0/99161187 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c02ed30 con 0x7f196c155b10 2024-03-29T04:08:52.465 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.484+0000 7ff5fbe4b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 -- 172.21.15.187:0/99161187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 msgr2=0x7f196c0b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 --2- 172.21.15.187:0/99161187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c0b4c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f196c0b7380 tx=0x7f195c02dbb0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 -- 172.21.15.187:0/99161187 shutdown_connections 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 --2- 172.21.15.187:0/99161187 >> v2:172.21.15.181:3300/0 conn(0x7f196c15fae0 0x7f196c0b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 --2- 172.21.15.187:0/99161187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c0b4c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 -- 172.21.15.187:0/99161187 >> 172.21.15.187:0/99161187 conn(0x7f196c0d1160 msgr2=0x7f196c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 -- 172.21.15.187:0/99161187 shutdown_connections 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.484+0000 7f19709e2640 1 -- 172.21.15.187:0/99161187 wait complete. 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.486+0000 7f19709e2640 1 Processor -- start 2024-03-29T04:08:52.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.486+0000 7f19709e2640 1 -- start start 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c0d03c0 0x7f196c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f196c15fae0 0x7f196c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f196c0b62d0 con 0x7f196c0d03c0 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f196c0b5ce0 con 0x7f196c155b10 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.487+0000 7f19709e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f196c0d0850 con 0x7f196c15fae0 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49278/0 (socket says 172.21.15.187:49278) 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 -- 172.21.15.187:0/2413200980 learned_addr learned my addr 172.21.15.187:0/2413200980 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f196ad9d640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f196c15fae0 0x7f196c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f196a59c640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c0d03c0 0x7f196c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.469 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 -- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f196c15fae0 msgr2=0x7f196c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f196c15fae0 0x7f196c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 -- 172.21.15.187:0/2413200980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c0d03c0 msgr2=0x7f196c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c0d03c0 0x7f196c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 -- 172.21.15.187:0/2413200980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c14a150 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1963fff640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f195400d7d0 tx=0x7f195400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954004030 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19540041d0 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c14a430 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540096a0 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c14a9f0 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.488+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1954009840 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.489+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f1954014850 con 0x7f196c155b10 2024-03-29T04:08:52.470 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.489+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c075890 con 0x7f196c155b10 2024-03-29T04:08:52.472 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.493+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1954009c70 con 0x7f196c155b10 2024-03-29T04:08:52.481 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.501+0000 7f9305e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T04:08:52.518 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.538+0000 7ff5fbe4b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:08:52.537 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.557+0000 7f9305e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T04:08:52.545 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.566+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d06b03de-42fc-4295-a039-9514a94cb7b2", "id": 2} v 0) -- 0x7f196c070200 con 0x7f196c155b10 2024-03-29T04:08:52.551 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.571+0000 7f1962ffd640 1 -- 172.21.15.187:0/2413200980 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d06b03de-42fc-4295-a039-9514a94cb7b2", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f1954014430 con 0x7f196c155b10 2024-03-29T04:08:52.551 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 msgr2=0x7f196c148360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f195400d7d0 tx=0x7f195400dc90 comp rx=0 tx=0).stop 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 shutdown_connections 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f196c15fae0 0x7f196c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f196c155b10 0x7f196c148360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 --2- 172.21.15.187:0/2413200980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f196c0d03c0 0x7f196c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 >> 172.21.15.187:0/2413200980 conn(0x7f196c0d1160 msgr2=0x7f196c0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.572+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 shutdown_connections 2024-03-29T04:08:52.553 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.573+0000 7f19709e2640 1 -- 172.21.15.187:0/2413200980 wait complete. 2024-03-29T04:08:52.563 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 659b7c6b-7766-443d-9547-d45aae9e9039 3 2024-03-29T04:08:52.563 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.584+0000 7ff5fbe4b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:08:52.586 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.606+0000 7f9305e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T04:08:52.617 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.637+0000 7ff5fbe4b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:08:52.643 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.663+0000 7f9305e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T04:08:52.659 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.679+0000 7ff5fbe4b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:08:52.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.684+0000 7f7ff68c3640 1 Processor -- start 2024-03-29T04:08:52.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.684+0000 7f7ff68c3640 1 -- start start 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ff00b37e0 0x7f7ff00b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 0x7f7ff00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 0x7f7ff006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff00b4100 con 0x7f7ff00cca80 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff00ce390 con 0x7f7ff00b37e0 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff68c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00ccfd0 con 0x7f7ff006da80 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ff00b37e0 0x7f7ff00b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff4e60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 0x7f7ff006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ff00b37e0 0x7f7ff00b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49292/0 (socket says 172.21.15.187:49292) 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 -- 172.21.15.187:0/129106621 learned_addr learned my addr 172.21.15.187:0/129106621 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7fef7fe640 1 --2- 172.21.15.187:0/129106621 >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 0x7f7ff00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 -- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 msgr2=0x7f7ff006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 0x7f7ff006df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 -- 172.21.15.187:0/129106621 >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 msgr2=0x7f7ff00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 --2- 172.21.15.187:0/129106621 >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 0x7f7ff00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7feffff640 1 -- 172.21.15.187:0/129106621 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff006bf90 con 0x7f7ff00b37e0 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7ff4e60640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 0x7f7ff006df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.685+0000 7f7fef7fe640 1 --2- 172.21.15.187:0/129106621 >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 0x7f7ff00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.686+0000 7f7feffff640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00b37e0 0x7f7ff00b57f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ff006d120 tx=0x7f7fe402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad2b3aeb2daeb2a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.686+0000 7f7feeffd640 1 -- 172.21.15.187:0/129106621 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4035020 con 0x7f7ff00b37e0 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.686+0000 7f7feeffd640 1 -- 172.21.15.187:0/129106621 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4031870 con 0x7f7ff00b37e0 2024-03-29T04:08:52.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.686+0000 7f7feeffd640 1 -- 172.21.15.187:0/129106621 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4031c40 con 0x7f7ff00b37e0 2024-03-29T04:08:52.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 -- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00b37e0 msgr2=0x7f7ff00b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.670 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00b37e0 0x7f7ff00b57f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ff006d120 tx=0x7f7fe402b910 comp rx=0 tx=0).stop 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 -- 172.21.15.187:0/129106621 shutdown_connections 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff006da80 0x7f7ff006df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/129106621 >> v2:172.21.15.181:3300/0 conn(0x7f7ff00cca80 0x7f7ff00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/129106621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00b37e0 0x7f7ff00b57f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 -- 172.21.15.187:0/129106621 >> 172.21.15.187:0/129106621 conn(0x7f7ff00cd310 msgr2=0x7f7ff00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 -- 172.21.15.187:0/129106621 shutdown_connections 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.687+0000 7f7ff68c3640 1 -- 172.21.15.187:0/129106621 wait complete. 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.688+0000 7f7ff68c3640 1 Processor -- start 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.688+0000 7f7ff68c3640 1 -- start start 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff006da80 0x7f7ff015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 0x7f7ff0147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff006c830 con 0x7f7ff006da80 2024-03-29T04:08:52.671 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff006c550 con 0x7f7ff00cca80 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff68c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff006cd50 con 0x7f7ff00b37e0 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 0x7f7ff0147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:41310/0 (socket says 172.21.15.187:41310) 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7fef7fe640 1 -- 172.21.15.187:0/1340324462 learned_addr learned my addr 172.21.15.187:0/1340324462 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7feffff640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff006da80 0x7f7ff015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 -- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 msgr2=0x7f7ff0146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 -- 172.21.15.187:0/1340324462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff006da80 msgr2=0x7f7ff015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff006da80 0x7f7ff015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 -- 172.21.15.187:0/1340324462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff0146fb0 con 0x7f7ff00cca80 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7fef7fe640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7ff4e60640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 0x7f7ff0147d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800efc0 tx=0x7f7fe800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.689+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8018030 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe800cd30 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8003bb0 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff0147290 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff0146970 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fe801b070 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f7fe801d940 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.690+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff0075890 con 0x7f7ff00cca80 2024-03-29T04:08:52.673 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.692+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7fe8010040 con 0x7f7ff00cca80 2024-03-29T04:08:52.688 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.708+0000 7f9305e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T04:08:52.745 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.765+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "659b7c6b-7766-443d-9547-d45aae9e9039", "id": 3} v 0) -- 0x7f7ff006bd50 con 0x7f7ff00cca80 2024-03-29T04:08:52.750 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7fecff9640 1 -- 172.21.15.187:0/1340324462 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "659b7c6b-7766-443d-9547-d45aae9e9039", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f7fe801de10 con 0x7f7ff00cca80 2024-03-29T04:08:52.751 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 msgr2=0x7f7ff0147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 0x7f7ff0147d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800efc0 tx=0x7f7fe800c490 comp rx=0 tx=0).stop 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 shutdown_connections 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ff00cca80 0x7f7ff0147d70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ff00b37e0 0x7f7ff0146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 --2- 172.21.15.187:0/1340324462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ff006da80 0x7f7ff015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.771+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 >> 172.21.15.187:0/1340324462 conn(0x7f7ff00cd310 msgr2=0x7f7ff00cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.772+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 shutdown_connections 2024-03-29T04:08:52.752 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.772+0000 7f7ff68c3640 1 -- 172.21.15.187:0/1340324462 wait complete. 2024-03-29T04:08:52.762 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 0a46197b-1e75-45b4-8166-96296c19c585 4 2024-03-29T04:08:52.775 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.796+0000 7ff5fbe4b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:08:52.811 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.831+0000 7f9305e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T04:08:52.844 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.865+0000 7ff5fbe4b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:08:52.885 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.905+0000 7f9305e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T04:08:52.890 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.910+0000 7ff5fbe4b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:08:52.902 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.922+0000 7f2b5b006640 1 Processor -- start 2024-03-29T04:08:52.902 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.922+0000 7f2b5b006640 1 -- start start 2024-03-29T04:08:52.903 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b5b006640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b540b4c90 0x7f2b5415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b5b006640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 0x7f2b540d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b5b006640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b540b5ff0 con 0x7f2b54155b10 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b5b006640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f2b54155ef0 con 0x7f2b540d03c0 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b5b006640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2b540b41b0 con 0x7f2b540b4c90 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b540b4c90 0x7f2b5415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b540b4c90 0x7f2b5415fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49316/0 (socket says 172.21.15.187:49316) 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 -- 172.21.15.187:0/264625609 learned_addr learned my addr 172.21.15.187:0/264625609 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b595a3640 1 --2- 172.21.15.187:0/264625609 >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 0x7f2b540d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 -- 172.21.15.187:0/264625609 >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 msgr2=0x7f2b540d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 --2- 172.21.15.187:0/264625609 >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 0x7f2b540d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 -- 172.21.15.187:0/264625609 >> v1:172.21.15.181:6789/0 conn(0x7f2b54155b10 legacy=0x7f2b540b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b53fff640 1 -- 172.21.15.187:0/264625609 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b540b4490 con 0x7f2b540b4c90 2024-03-29T04:08:52.904 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.923+0000 7f2b595a3640 1 --2- 172.21.15.187:0/264625609 >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 0x7f2b540d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.924+0000 7f2b53fff640 1 --2- 172.21.15.187:0/264625609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b540b4c90 0x7f2b5415fae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406d650 tx=0x7f2b4002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a9abdca0015263 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.924+0000 7f2b537fe640 1 -- 172.21.15.187:0/264625609 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40034070 con 0x7f2b540b4c90 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.924+0000 7f2b537fe640 1 -- 172.21.15.187:0/264625609 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40030860 con 0x7f2b540b4c90 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.924+0000 7f2b537fe640 1 -- 172.21.15.187:0/264625609 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40030c30 con 0x7f2b540b4c90 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 -- 172.21.15.187:0/264625609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b540b4c90 msgr2=0x7f2b5415fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 --2- 172.21.15.187:0/264625609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b540b4c90 0x7f2b5415fae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406d650 tx=0x7f2b4002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 -- 172.21.15.187:0/264625609 shutdown_connections 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 --2- 172.21.15.187:0/264625609 >> v2:172.21.15.181:3300/0 conn(0x7f2b540d03c0 0x7f2b540d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 --2- 172.21.15.187:0/264625609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b540b4c90 0x7f2b5415fae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.906 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 -- 172.21.15.187:0/264625609 >> 172.21.15.187:0/264625609 conn(0x7f2b540d1160 msgr2=0x7f2b540d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.907 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 -- 172.21.15.187:0/264625609 shutdown_connections 2024-03-29T04:08:52.907 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.925+0000 7f2b5b006640 1 -- 172.21.15.187:0/264625609 wait complete. 2024-03-29T04:08:52.907 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.926+0000 7f2b5b006640 1 Processor -- start 2024-03-29T04:08:52.907 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.926+0000 7f2b5b006640 1 -- start start 2024-03-29T04:08:52.909 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b540b4c90 0x7f2b5413dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.909 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 0x7f2b5415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.909 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 0x7f2b5415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5415b9d0 con 0x7f2b540b4c90 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b540b62d0 con 0x7f2b54155b10 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b5b006640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5415bb50 con 0x7f2b540d03c0 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b595a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 0x7f2b5415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b53fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 0x7f2b5415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b595a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 0x7f2b5415e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49322/0 (socket says 172.21.15.187:49322) 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b595a3640 1 -- 172.21.15.187:0/3804214188 learned_addr learned my addr 172.21.15.187:0/3804214188 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b58da2640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b540b4c90 0x7f2b5413dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b53fff640 1 -- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 msgr2=0x7f2b5415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b53fff640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 0x7f2b5415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b53fff640 1 -- 172.21.15.187:0/3804214188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b540b4c90 msgr2=0x7f2b5413dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.927+0000 7f2b53fff640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b540b4c90 0x7f2b5413dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.910 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b53fff640 1 -- 172.21.15.187:0/3804214188 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5414d350 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b53fff640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 0x7f2b5415dce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b40033000 tx=0x7f2b40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40048070 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40030660 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40034040 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5414d630 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5414dbf0 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b4004b070 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.928+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f2b40043920 con 0x7f2b540d03c0 2024-03-29T04:08:52.911 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.929+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b54075890 con 0x7f2b540d03c0 2024-03-29T04:08:52.912 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:52.931+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2b40052070 con 0x7f2b540d03c0 2024-03-29T04:08:52.934 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:52.954+0000 7f9305e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T04:08:52.942 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:52.962+0000 7ff5fbe4b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:08:52.983 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.004+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0a46197b-1e75-45b4-8166-96296c19c585", "id": 4} v 0) -- 0x7f2b540b5940 con 0x7f2b540d03c0 2024-03-29T04:08:52.988 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.008+0000 7f9305e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T04:08:52.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.009+0000 7f2b517fa640 1 -- 172.21.15.187:0/3804214188 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0a46197b-1e75-45b4-8166-96296c19c585", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f2b40030830 con 0x7f2b540d03c0 2024-03-29T04:08:52.989 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-03-29T04:08:52.989 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.010+0000 7ff5fbe4b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:08:52.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 msgr2=0x7f2b5415dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 0x7f2b5415dce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2b40033000 tx=0x7f2b40004730 comp rx=0 tx=0).stop 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 shutdown_connections 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b54155b10 0x7f2b5415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b540d03c0 0x7f2b5415dce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 --2- 172.21.15.187:0/3804214188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b540b4c90 0x7f2b5413dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 >> 172.21.15.187:0/3804214188 conn(0x7f2b540d1160 msgr2=0x7f2b5415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 shutdown_connections 2024-03-29T04:08:52.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.010+0000 7f2b5b006640 1 -- 172.21.15.187:0/3804214188 wait complete. 2024-03-29T04:08:53.000 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new ec789e34-2b4b-46fa-8f0f-37f0ddc319ca 5 2024-03-29T04:08:53.037 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.057+0000 7f9305e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T04:08:53.138 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.158+0000 7f8620e70640 1 Processor -- start 2024-03-29T04:08:53.139 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f8620e70640 1 -- start start 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f8620e70640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f861c0b43c0 0x7f861c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f8620e70640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861c162ac0 con 0x7f861c162150 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f8620e70640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861c162da0 con 0x7f861c0d03f0 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f8620e70640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f861c15eae0 con 0x7f861c0b43c0 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f861ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f861c0b43c0 0x7f861c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f861ad9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f861c0b43c0 0x7f861c15ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49336/0 (socket says 172.21.15.187:49336) 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.159+0000 7f861ad9d640 1 -- 172.21.15.187:0/1862513387 learned_addr learned my addr 172.21.15.187:0/1862513387 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.160+0000 7f861ad9d640 1 -- 172.21.15.187:0/1862513387 >> v1:172.21.15.187:6789/0 conn(0x7f861c0d03f0 legacy=0x7f861c15f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.160+0000 7f861ad9d640 1 -- 172.21.15.187:0/1862513387 >> v1:172.21.15.181:6789/0 conn(0x7f861c162150 legacy=0x7f861c0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.160+0000 7f861ad9d640 1 -- 172.21.15.187:0/1862513387 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c15ed00 con 0x7f861c0b43c0 2024-03-29T04:08:53.141 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.160+0000 7f861ad9d640 1 --2- 172.21.15.187:0/1862513387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0b43c0 0x7f861c15ff30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8614009e50 tx=0x7f861402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c4f3da3bb9b5ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.141 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8619d9b640 1 -- 172.21.15.187:0/1862513387 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861402ca80 con 0x7f861c0b43c0 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8619d9b640 1 -- 172.21.15.187:0/1862513387 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861402cc20 con 0x7f861c0b43c0 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8619d9b640 1 -- 172.21.15.187:0/1862513387 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140343f0 con 0x7f861c0b43c0 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 -- 172.21.15.187:0/1862513387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0b43c0 msgr2=0x7f861c15ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 --2- 172.21.15.187:0/1862513387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0b43c0 0x7f861c15ff30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8614009e50 tx=0x7f861402bee0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 -- 172.21.15.187:0/1862513387 shutdown_connections 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 --2- 172.21.15.187:0/1862513387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0b43c0 0x7f861c15ff30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 -- 172.21.15.187:0/1862513387 >> 172.21.15.187:0/1862513387 conn(0x7f861c0672c0 msgr2=0x7f861c0676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 -- 172.21.15.187:0/1862513387 shutdown_connections 2024-03-29T04:08:53.142 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.161+0000 7f8620e70640 1 -- 172.21.15.187:0/1862513387 wait complete. 2024-03-29T04:08:53.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.162+0000 7f8620e70640 1 Processor -- start 2024-03-29T04:08:53.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 -- start start 2024-03-29T04:08:53.145 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b43c0 0x7f861c146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 0x7f861c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f861c163080 con 0x7f861c0b43c0 2024-03-29T04:08:53.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f861c0cfb90 con 0x7f861c0d03f0 2024-03-29T04:08:53.146 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8620e70640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f861c15f070 con 0x7f861c162150 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f861ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 0x7f861c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8613fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f861ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 0x7f861c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:41336/0 (socket says 172.21.15.187:41336) 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f8613fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49346/0 (socket says 172.21.15.187:49346) 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f861ad9d640 1 -- 172.21.15.187:0/784877292 learned_addr learned my addr 172.21.15.187:0/784877292 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.163+0000 7f861a59c640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b43c0 0x7f861c146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 -- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 msgr2=0x7f861c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 0x7f861c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 -- 172.21.15.187:0/784877292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b43c0 msgr2=0x7f861c146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.147 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b43c0 0x7f861c146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 -- 172.21.15.187:0/784877292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c0b0580 con 0x7f861c0d03f0 2024-03-29T04:08:53.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8613fff640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f860400c340 tx=0x7f860400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86040052d0 con 0x7f861c0d03f0 2024-03-29T04:08:53.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8604005490 con 0x7f861c0d03f0 2024-03-29T04:08:53.148 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c0b0860 con 0x7f861c0d03f0 2024-03-29T04:08:53.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604007260 con 0x7f861c0d03f0 2024-03-29T04:08:53.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c0b0e20 con 0x7f861c0d03f0 2024-03-29T04:08:53.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8604007400 con 0x7f861c0d03f0 2024-03-29T04:08:53.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.164+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f8604011590 con 0x7f861c0d03f0 2024-03-29T04:08:53.149 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.165+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c075890 con 0x7f861c0d03f0 2024-03-29T04:08:53.150 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.167+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8604003a50 con 0x7f861c0d03f0 2024-03-29T04:08:53.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.237+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec789e34-2b4b-46fa-8f0f-37f0ddc319ca", "id": 5} v 0) -- 0x7f861c0b8170 con 0x7f861c0d03f0 2024-03-29T04:08:53.223 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.243+0000 7f8612ffd640 1 -- 172.21.15.187:0/784877292 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec789e34-2b4b-46fa-8f0f-37f0ddc319ca", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8604003c30 con 0x7f861c0d03f0 2024-03-29T04:08:53.223 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 msgr2=0x7f861c0afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f860400c340 tx=0x7f860400c800 comp rx=0 tx=0).stop 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 shutdown_connections 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f861c162150 0x7f861c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f861c0d03f0 0x7f861c0afa30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 --2- 172.21.15.187:0/784877292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b43c0 0x7f861c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 >> 172.21.15.187:0/784877292 conn(0x7f861c0672c0 msgr2=0x7f861c15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.225 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 shutdown_connections 2024-03-29T04:08:53.225 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.244+0000 7f8620e70640 1 -- 172.21.15.187:0/784877292 wait complete. 2024-03-29T04:08:53.234 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 1e1cb5e4-a484-4e65-b665-4335faa462ff 6 2024-03-29T04:08:53.335 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.355+0000 7f5dc3d9d640 1 Processor -- start 2024-03-29T04:08:53.335 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 -- start start 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 0x7f5dbc0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dbc0b5ff0 con 0x7f5dbc155b10 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5dbc155ef0 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc3d9d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5dbc0b41b0 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.356+0000 7f5dc1338640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49362/0 (socket says 172.21.15.187:49362) 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 -- 172.21.15.187:0/2556527308 learned_addr learned my addr 172.21.15.187:0/2556527308 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc233a640 1 --2- 172.21.15.187:0/2556527308 >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 0x7f5dbc0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 -- 172.21.15.187:0/2556527308 >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 msgr2=0x7f5dbc0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.337 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 --2- 172.21.15.187:0/2556527308 >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 0x7f5dbc0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.338 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 -- 172.21.15.187:0/2556527308 >> v1:172.21.15.187:6789/0 conn(0x7f5dbc155b10 legacy=0x7f5dbc0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.338 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc1338640 1 -- 172.21.15.187:0/2556527308 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc0b4490 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.338 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.357+0000 7f5dc233a640 1 --2- 172.21.15.187:0/2556527308 >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 0x7f5dbc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc1338640 1 --2- 172.21.15.187:0/2556527308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06d650 tx=0x7f5dac02e170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8e0072f54ad076 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc0b37640 1 -- 172.21.15.187:0/2556527308 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03b070 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc0b37640 1 -- 172.21.15.187:0/2556527308 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac036440 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc0b37640 1 -- 172.21.15.187:0/2556527308 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac036810 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/2556527308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0b4c90 msgr2=0x7f5dbc15fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/2556527308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06d650 tx=0x7f5dac02e170 comp rx=0 tx=0).stop 2024-03-29T04:08:53.339 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/2556527308 shutdown_connections 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/2556527308 >> v2:172.21.15.181:3300/0 conn(0x7f5dbc0d03c0 0x7f5dbc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/2556527308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0b4c90 0x7f5dbc15fae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/2556527308 >> 172.21.15.187:0/2556527308 conn(0x7f5dbc0d1160 msgr2=0x7f5dbc0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/2556527308 shutdown_connections 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.358+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/2556527308 wait complete. 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 Processor -- start 2024-03-29T04:08:53.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 -- start start 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 0x7f5dbc0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc155b10 0x7f5dbc148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc15b9d0 con 0x7f5dbc155b10 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc0b62d0 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.342 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc3d9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc15bb50 con 0x7f5dbc0b4c90 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc1338640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc1b39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 0x7f5dbc0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc1338640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49376/0 (socket says 172.21.15.187:49376) 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc1338640 1 -- 172.21.15.187:0/4187770074 learned_addr learned my addr 172.21.15.187:0/4187770074 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.360+0000 7f5dc233a640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc155b10 0x7f5dbc148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 -- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 msgr2=0x7f5dbc0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 0x7f5dbc0cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.343 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 -- 172.21.15.187:0/4187770074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc155b10 msgr2=0x7f5dbc148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc155b10 0x7f5dbc148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 -- 172.21.15.187:0/4187770074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc148950 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1b39640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 0x7f5dbc0cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc1338640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02e1b0 tx=0x7f5dac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac046070 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac03d050 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac03b030 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc15cb60 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc148ed0 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5dac049070 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.361+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5dac041a30 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.344 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.362+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc075890 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.345 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.364+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5dac050020 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.367 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.388+0000 7ff5fbe4b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:08:53.417 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.437+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1e1cb5e4-a484-4e65-b665-4335faa462ff", "id": 6} v 0) -- 0x7f5dbc070ab0 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.422 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.443+0000 7f5db27fc640 1 -- 172.21.15.187:0/4187770074 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1e1cb5e4-a484-4e65-b665-4335faa462ff", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5dac041440 con 0x7f5dbc0d03c0 2024-03-29T04:08:53.423 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 msgr2=0x7f5dbc140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5dac02e1b0 tx=0x7f5dac0046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 shutdown_connections 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc155b10 0x7f5dbc148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc0d03c0 0x7f5dbc140e10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 --2- 172.21.15.187:0/4187770074 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc0b4c90 0x7f5dbc0cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 >> 172.21.15.187:0/4187770074 conn(0x7f5dbc0d1160 msgr2=0x7f5dbc06fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.424 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 shutdown_connections 2024-03-29T04:08:53.425 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.444+0000 7f5dc3d9d640 1 -- 172.21.15.187:0/4187770074 wait complete. 2024-03-29T04:08:53.434 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 41add9a8-91cc-4862-a9a6-f9b397b77cc9 7 2024-03-29T04:08:53.440 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.461+0000 7f9305e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T04:08:53.502 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.522+0000 7ff5fbe4b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:08:53.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.554+0000 7f69418d3640 1 Processor -- start 2024-03-29T04:08:53.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.554+0000 7f69418d3640 1 -- start start 2024-03-29T04:08:53.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f69418d3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f693c06bc90 0x7f693c15ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f69418d3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c15c9f0 con 0x7f693c159f90 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f69418d3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0ce770 con 0x7f693c0ce310 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f69418d3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f693c160520 con 0x7f693c06bc90 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f693a7fc640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f693c0ce310 0x7f693c06b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:36808/0 (socket says 172.21.15.187:36808) 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.555+0000 7f693a7fc640 1 -- 172.21.15.187:0/3891178946 learned_addr learned my addr 172.21.15.187:0/3891178946 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f693b7fe640 1 --2- 172.21.15.187:0/3891178946 >> v2:172.21.15.181:3300/0 conn(0x7f693c06bc90 0x7f693c15ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f6939ffb640 1 -- 172.21.15.187:0/3891178946 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050899703 0 0) 0x7f693c0ce770 con 0x7f693c0ce310 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f6939ffb640 1 -- 172.21.15.187:0/3891178946 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6924003520 con 0x7f693c0ce310 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f693b7fe640 1 -- 172.21.15.187:0/3891178946 >> v1:172.21.15.187:6789/0 conn(0x7f693c0ce310 legacy=0x7f693c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f693b7fe640 1 -- 172.21.15.187:0/3891178946 >> v1:172.21.15.181:6789/0 conn(0x7f693c159f90 legacy=0x7f693c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.556+0000 7f693b7fe640 1 -- 172.21.15.187:0/3891178946 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c160750 con 0x7f693c06bc90 2024-03-29T04:08:53.537 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f693b7fe640 1 --2- 172.21.15.187:0/3891178946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c06bc90 0x7f693c15ffe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d90 tx=0x7f693402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983bfa8cb28efb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.537 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f6939ffb640 1 -- 172.21.15.187:0/3891178946 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934039070 con 0x7f693c06bc90 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f6939ffb640 1 -- 172.21.15.187:0/3891178946 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693402cd00 con 0x7f693c06bc90 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f6939ffb640 1 -- 172.21.15.187:0/3891178946 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934034640 con 0x7f693c06bc90 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 -- 172.21.15.187:0/3891178946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c06bc90 msgr2=0x7f693c15ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 --2- 172.21.15.187:0/3891178946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c06bc90 0x7f693c15ffe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6934009d90 tx=0x7f693402bf50 comp rx=0 tx=0).stop 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 -- 172.21.15.187:0/3891178946 shutdown_connections 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 --2- 172.21.15.187:0/3891178946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c06bc90 0x7f693c15ffe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 -- 172.21.15.187:0/3891178946 >> 172.21.15.187:0/3891178946 conn(0x7f693c0d1160 msgr2=0x7f693c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 -- 172.21.15.187:0/3891178946 shutdown_connections 2024-03-29T04:08:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.557+0000 7f69418d3640 1 -- 172.21.15.187:0/3891178946 wait complete. 2024-03-29T04:08:53.539 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.558+0000 7f69418d3640 1 Processor -- start 2024-03-29T04:08:53.539 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 -- start start 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c0ce310 0x7f693c1f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c159f90 0x7f693c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f693c160d10 con 0x7f693c0ce310 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f693c15cd00 con 0x7f693c06bc90 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f69418d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f693c15a370 con 0x7f693c159f90 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.559+0000 7f693affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c159f90 0x7f693c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:49386/0 (socket says 172.21.15.187:49386) 2024-03-29T04:08:53.541 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 -- 172.21.15.187:0/4030354294 learned_addr learned my addr 172.21.15.187:0/4030354294 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693a7fc640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c0ce310 0x7f693c1f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 -- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c159f90 msgr2=0x7f693c1f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c159f90 0x7f693c1f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 -- 172.21.15.187:0/4030354294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c0ce310 msgr2=0x7f693c1f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c0ce310 0x7f693c1f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 -- 172.21.15.187:0/4030354294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c155500 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f693affd640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f693000c3f0 tx=0x7f693000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69300052d0 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6930005490 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1f5da0 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930007260 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c0a9630 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6930007400 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.560+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f6930011bf0 con 0x7f693c06bc90 2024-03-29T04:08:53.542 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.561+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c075890 con 0x7f693c06bc90 2024-03-29T04:08:53.543 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.563+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f693c15c9f0 con 0x7f693c06bc90 2024-03-29T04:08:53.583 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.603+0000 7f9305e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T04:08:53.615 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.635+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "41add9a8-91cc-4862-a9a6-f9b397b77cc9", "id": 7} v 0) -- 0x7f693c0735a0 con 0x7f693c06bc90 2024-03-29T04:08:53.623 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.643+0000 7f6923fff640 1 -- 172.21.15.187:0/4030354294 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41add9a8-91cc-4862-a9a6-f9b397b77cc9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f69300035f0 con 0x7f693c06bc90 2024-03-29T04:08:53.623 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-03-29T04:08:53.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 msgr2=0x7f693c154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f693000c3f0 tx=0x7f693000c8b0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 shutdown_connections 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f693c159f90 0x7f693c1f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f693c0ce310 0x7f693c1f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 --2- 172.21.15.187:0/4030354294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f693c06bc90 0x7f693c154fc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 >> 172.21.15.187:0/4030354294 conn(0x7f693c0d1160 msgr2=0x7f693c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 shutdown_connections 2024-03-29T04:08:53.625 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:08:53.644+0000 7f69418d3640 1 -- 172.21.15.187:0/4030354294 wait complete. 2024-03-29T04:08:53.625 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.646+0000 7ff5fbe4b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:08:53.634 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T04:08:53.635 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T04:08:53.637 INFO:tasks.ceph.osd.0:Started 2024-03-29T04:08:53.637 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T04:08:53.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T04:08:53.641 INFO:tasks.ceph.osd.1:Started 2024-03-29T04:08:53.641 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T04:08:53.641 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T04:08:53.644 INFO:tasks.ceph.osd.2:Started 2024-03-29T04:08:53.644 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T04:08:53.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T04:08:53.646 INFO:tasks.ceph.osd.3:Started 2024-03-29T04:08:53.646 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T04:08:53.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T04:08:53.677 INFO:tasks.ceph.osd.4:Started 2024-03-29T04:08:53.677 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T04:08:53.677 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T04:08:53.679 INFO:tasks.ceph.osd.5:Started 2024-03-29T04:08:53.679 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T04:08:53.680 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T04:08:53.681 INFO:tasks.ceph.osd.6:Started 2024-03-29T04:08:53.682 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T04:08:53.682 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T04:08:53.684 INFO:tasks.ceph.osd.7:Started 2024-03-29T04:08:53.684 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:53.685 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.706+0000 7ff5fbe4b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:08:53.733 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.753+0000 7f9305e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T04:08:53.788 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.808+0000 7f9305e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.814+0000 7f6b27dbd640 1 Processor -- start 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 -- start start 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b20160540 0x7f6b200d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b200b5f80 con 0x7f6b200b5aa0 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6b200b6260 con 0x7f6b20160540 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.815+0000 7f6b27dbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b200b6540 con 0x7f6b20145090 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b2635a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b20160540 0x7f6b200d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:52194/0 (socket says 172.21.15.181:52194) 2024-03-29T04:08:53.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 -- 172.21.15.181:0/1550400240 learned_addr learned my addr 172.21.15.181:0/1550400240 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:08:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 -- 172.21.15.181:0/1550400240 >> v2:172.21.15.187:3300/0 conn(0x7f6b20160540 msgr2=0x7f6b200d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 --2- 172.21.15.181:0/1550400240 >> v2:172.21.15.187:3300/0 conn(0x7f6b20160540 0x7f6b200d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 -- 172.21.15.181:0/1550400240 >> v1:172.21.15.187:6789/0 conn(0x7f6b200b5aa0 legacy=0x7f6b2015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.816+0000 7f6b25b59640 1 -- 172.21.15.181:0/1550400240 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b200b6820 con 0x7f6b20145090 2024-03-29T04:08:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b25b59640 1 --2- 172.21.15.181:0/1550400240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009870 tx=0x7f6b1002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58ef011c02db4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b24b57640 1 -- 172.21.15.181:0/1550400240 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10035020 con 0x7f6b20145090 2024-03-29T04:08:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b24b57640 1 -- 172.21.15.181:0/1550400240 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b10031640 con 0x7f6b20145090 2024-03-29T04:08:53.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b24b57640 1 -- 172.21.15.181:0/1550400240 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1002d040 con 0x7f6b20145090 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 -- 172.21.15.181:0/1550400240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 msgr2=0x7f6b200b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/1550400240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009870 tx=0x7f6b1002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 -- 172.21.15.181:0/1550400240 shutdown_connections 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/1550400240 >> v2:172.21.15.187:3300/0 conn(0x7f6b20160540 0x7f6b200d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/1550400240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b20145090 0x7f6b200b1720 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.817+0000 7f6b27dbd640 1 -- 172.21.15.181:0/1550400240 >> 172.21.15.181:0/1550400240 conn(0x7f6b2006f8b0 msgr2=0x7f6b2006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.818+0000 7f6b27dbd640 1 -- 172.21.15.181:0/1550400240 shutdown_connections 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.818+0000 7f6b27dbd640 1 -- 172.21.15.181:0/1550400240 wait complete. 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 Processor -- start 2024-03-29T04:08:53.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 -- start start 2024-03-29T04:08:53.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 0x7f6b2006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b20160540 0x7f6b2013b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20145470 con 0x7f6b20145090 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20146000 con 0x7f6b20160540 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.819+0000 7f6b27dbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b201455f0 con 0x7f6b200b5aa0 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25358640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 0x7f6b2006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b2635a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b20160540 0x7f6b2013b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25358640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 0x7f6b2006d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59864/0 (socket says 172.21.15.181:59864) 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25358640 1 -- 172.21.15.181:0/2787395842 learned_addr learned my addr 172.21.15.181:0/2787395842 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:52208/0 (socket says 172.21.15.181:52208) 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 -- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b20160540 msgr2=0x7f6b2013b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b20160540 0x7f6b2013b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 -- 172.21.15.181:0/2787395842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 msgr2=0x7f6b2006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 0x7f6b2006d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 -- 172.21.15.181:0/2787395842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b201f7740 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b25b59640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b10033000 tx=0x7f6b100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10040070 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b100399f0 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.820+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10035040 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.821+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2006db20 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.821+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b200b25e0 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.821+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b10038020 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.821+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6b10044410 con 0x7f6b200b5aa0 2024-03-29T04:08:53.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.822+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b20075a10 con 0x7f6b200b5aa0 2024-03-29T04:08:53.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.824+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6b1003a7b0 con 0x7f6b200b5aa0 2024-03-29T04:08:53.824 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.844+0000 7ff5fbe4b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:08:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.884+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b200d2c00 con 0x7f6b200b5aa0 2024-03-29T04:08:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b167fc640 1 -- 172.21.15.181:0/2787395842 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6b1003a990 con 0x7f6b200b5aa0 2024-03-29T04:08:53.864 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:08:53.865 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":10,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:08:53.637717+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:53.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 msgr2=0x7f6b201f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b10033000 tx=0x7f6b100046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 shutdown_connections 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b20160540 0x7f6b2013b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b20145090 0x7f6b2006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 --2- 172.21.15.181:0/2787395842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b200b5aa0 0x7f6b201f7200 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.885+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 >> 172.21.15.181:0/2787395842 conn(0x7f6b2006f8b0 msgr2=0x7f6b200b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.886+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 shutdown_connections 2024-03-29T04:08:53.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:08:53.886+0000 7f6b27dbd640 1 -- 172.21.15.181:0/2787395842 wait complete. 2024-03-29T04:08:53.876 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T04:08:53.876 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T04:08:53.901 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:53.921+0000 7f9305e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:08:53.922 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:53.943+0000 7ff5fbe4b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:08:53.994 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:54.014+0000 7f9305e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T04:08:54.078 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:54.098+0000 7ff5fbe4b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:08:54.120 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:54.140+0000 7ff5fbe4b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:08:54.151 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:54.171+0000 7f9305e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T04:08:54.193 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:54.213+0000 7f9305e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T04:08:54.240 INFO:tasks.ceph.osd.0.smithi181.stderr:2024-03-29T04:08:54.260+0000 7f12dec15640 -1 Falling back to public interface 2024-03-29T04:08:54.245 INFO:tasks.ceph.osd.1.smithi181.stderr:2024-03-29T04:08:54.265+0000 7fb042a1d640 -1 Falling back to public interface 2024-03-29T04:08:54.257 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-03-29T04:08:54.277+0000 7f4a3841b640 -1 Falling back to public interface 2024-03-29T04:08:54.265 INFO:tasks.ceph.osd.3.smithi181.stderr:2024-03-29T04:08:54.285+0000 7f181f993640 -1 Falling back to public interface 2024-03-29T04:08:54.278 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:54.298+0000 7ff5fbe4b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:08:54.302 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-03-29T04:08:54.322+0000 7f0992138640 -1 Falling back to public interface 2024-03-29T04:08:54.303 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-03-29T04:08:54.323+0000 7f4618d28640 -1 Falling back to public interface 2024-03-29T04:08:54.305 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-03-29T04:08:54.325+0000 7f6e34619640 -1 Falling back to public interface 2024-03-29T04:08:54.306 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-03-29T04:08:54.326+0000 7f9d6c0cc640 -1 Falling back to public interface 2024-03-29T04:08:54.354 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:54.375+0000 7f9305e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T04:08:54.682 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:08:54.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.793+0000 7f1de3347640 1 Processor -- start 2024-03-29T04:08:54.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.793+0000 7f1de3347640 1 -- start start 2024-03-29T04:08:54.774 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de3347640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 0x7f1ddc06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:54.774 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de3347640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ddc15c4f0 0x7f1ddc06d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:54.774 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de3347640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ddc163230 con 0x7f1ddc15ef40 2024-03-29T04:08:54.774 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de3347640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f1ddc162490 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.774 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de3347640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1ddc162770 con 0x7f1ddc15c110 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ddc15c4f0 0x7f1ddc06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de18e4640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1ddc15ef40 0x7f1ddc15f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37232/0 (socket says 172.21.15.181:37232) 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ddc15c4f0 0x7f1ddc06d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59874/0 (socket says 172.21.15.181:59874) 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de18e4640 1 -- 172.21.15.181:0/1038084079 learned_addr learned my addr 172.21.15.181:0/1038084079 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de10e3640 1 --2- 172.21.15.181:0/1038084079 >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 0x7f1ddc06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 -- 172.21.15.181:0/1038084079 >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 msgr2=0x7f1ddc06cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 --2- 172.21.15.181:0/1038084079 >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 0x7f1ddc06cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 -- 172.21.15.181:0/1038084079 >> v1:172.21.15.181:6789/0 conn(0x7f1ddc15ef40 legacy=0x7f1ddc15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de08e2640 1 -- 172.21.15.181:0/1038084079 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc162a50 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1dd3fff640 1 -- 172.21.15.181:0/1038084079 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230884232 0 0) 0x7f1ddc163230 con 0x7f1ddc15ef40 2024-03-29T04:08:54.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.794+0000 7f1de10e3640 1 --2- 172.21.15.181:0/1038084079 >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 0x7f1ddc06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.795+0000 7f1de08e2640 1 --2- 172.21.15.181:0/1038084079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc06d520 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80097a0 tx=0x7f1dd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b726f72be1cd085 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.795+0000 7f1dd3fff640 1 -- 172.21.15.181:0/1038084079 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80315a0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.795+0000 7f1dd3fff640 1 -- 172.21.15.181:0/1038084079 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd8037070 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.795+0000 7f1dd3fff640 1 -- 172.21.15.181:0/1038084079 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8031960 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 -- 172.21.15.181:0/1038084079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 msgr2=0x7f1ddc06d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 --2- 172.21.15.181:0/1038084079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc06d520 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80097a0 tx=0x7f1dd802b840 comp rx=0 tx=0).stop 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 -- 172.21.15.181:0/1038084079 shutdown_connections 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 --2- 172.21.15.181:0/1038084079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc06d520 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 --2- 172.21.15.181:0/1038084079 >> v2:172.21.15.187:3300/0 conn(0x7f1ddc15c110 0x7f1ddc06cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 -- 172.21.15.181:0/1038084079 >> 172.21.15.181:0/1038084079 conn(0x7f1ddc0d2250 msgr2=0x7f1ddc0d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 -- 172.21.15.181:0/1038084079 shutdown_connections 2024-03-29T04:08:54.777 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.796+0000 7f1de3347640 1 -- 172.21.15.181:0/1038084079 wait complete. 2024-03-29T04:08:54.778 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.797+0000 7f1de3347640 1 Processor -- start 2024-03-29T04:08:54.778 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.797+0000 7f1de3347640 1 -- start start 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ddc15c110 0x7f1ddc0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ddc15ef40 0x7f1ddc153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0cede0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0cec60 con 0x7f1ddc15c110 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de3347640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ddc0cef60 con 0x7f1ddc15ef40 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59880/0 (socket says 172.21.15.181:59880) 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 -- 172.21.15.181:0/2896202346 learned_addr learned my addr 172.21.15.181:0/2896202346 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de10e3640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ddc15c110 0x7f1ddc0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de18e4640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ddc15ef40 0x7f1ddc153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 -- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ddc15ef40 msgr2=0x7f1ddc153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ddc15ef40 0x7f1ddc153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.780 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 -- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ddc15c110 msgr2=0x7f1ddc0af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ddc15c110 0x7f1ddc0af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 -- 172.21.15.181:0/2896202346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc153cd0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1de08e2640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80098c0 tx=0x7f1dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.798+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8032630 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd80327d0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ddc0afef0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8032ba0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ddc0b04b0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1dd804e7c0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.799+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1dd802f030 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.781 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.800+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ddc0758d0 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.782 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.802+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1dd8040070 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.843 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.864+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ddc06f880 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.844 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.864+0000 7f1dd1ffb640 1 -- 172.21.15.181:0/2896202346 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1dd8052020 con 0x7f1ddc15c4f0 2024-03-29T04:08:54.844 INFO:teuthology.misc.health.smithi181.stdout: 2024-03-29T04:08:54.844 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":10,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:08:53.637717+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 msgr2=0x7f1ddc0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80098c0 tx=0x7f1dd80046c0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 shutdown_connections 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ddc15ef40 0x7f1ddc153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ddc15c4f0 0x7f1ddc0b3190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.845 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 --2- 172.21.15.181:0/2896202346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ddc15c110 0x7f1ddc0af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:08:54.846 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 >> 172.21.15.181:0/2896202346 conn(0x7f1ddc0d2250 msgr2=0x7f1ddc15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:08:54.846 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 shutdown_connections 2024-03-29T04:08:54.846 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:08:54.865+0000 7f1de3347640 1 -- 172.21.15.181:0/2896202346 wait complete. 2024-03-29T04:08:54.854 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:08:55.098 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:55.119+0000 7ff5fbe4b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:08:55.375 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:55.396+0000 7ff5fbe4b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:08:55.427 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:55.448+0000 7ff5fbe4b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:08:55.437 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:55.457+0000 7f9305e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T04:08:55.542 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:55.563+0000 7ff5fbe4b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:08:55.717 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:55.738+0000 7f9305e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T04:08:55.769 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:55.789+0000 7f9305e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T04:08:55.883 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-03-29T04:08:55.903+0000 7f9305e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T04:08:56.555 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:56.575+0000 7ff59cb70640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T04:08:56.555 INFO:tasks.ceph.mgr.y.smithi181.stderr:2024-03-29T04:08:56.576+0000 7ff59cb70640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T04:08:57.751 INFO:tasks.ceph.osd.1.smithi181.stderr:2024-03-29T04:08:57.771+0000 7fb042a1d640 -1 osd.1 0 log_to_monitors true 2024-03-29T04:08:57.751 INFO:tasks.ceph.osd.0.smithi181.stderr:2024-03-29T04:08:57.772+0000 7f12dec15640 -1 osd.0 0 log_to_monitors true 2024-03-29T04:08:57.767 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-03-29T04:08:57.787+0000 7f0992138640 -1 osd.4 0 log_to_monitors true 2024-03-29T04:08:57.796 INFO:tasks.ceph.osd.3.smithi181.stderr:2024-03-29T04:08:57.816+0000 7f181f993640 -1 osd.3 0 log_to_monitors true 2024-03-29T04:08:57.797 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-03-29T04:08:57.818+0000 7f4a3841b640 -1 osd.2 0 log_to_monitors true 2024-03-29T04:08:57.820 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-03-29T04:08:57.841+0000 7f9d6c0cc640 -1 osd.5 0 log_to_monitors true 2024-03-29T04:08:57.826 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-03-29T04:08:57.847+0000 7f4618d28640 -1 osd.6 0 log_to_monitors true 2024-03-29T04:08:57.827 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-03-29T04:08:57.847+0000 7f6e34619640 -1 osd.7 0 log_to_monitors true 2024-03-29T04:09:01.565 INFO:tasks.ceph.osd.0.smithi181.stderr:2024-03-29T04:09:01.583+0000 7f12da5db640 -1 osd.0 0 waiting for initial osdmap 2024-03-29T04:09:01.582 INFO:tasks.ceph.osd.1.smithi181.stderr:2024-03-29T04:09:01.602+0000 7fb03e2ed640 -1 osd.1 0 waiting for initial osdmap 2024-03-29T04:09:01.582 INFO:tasks.ceph.osd.0.smithi181.stderr:2024-03-29T04:09:01.603+0000 7f12d23fb640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.594 INFO:tasks.ceph.osd.1.smithi181.stderr:2024-03-29T04:09:01.615+0000 7fb03611e640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.662 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:01.671 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-03-29T04:09:01.691+0000 7f4614cbe640 -1 osd.6 0 waiting for initial osdmap 2024-03-29T04:09:01.684 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-03-29T04:09:01.703+0000 7f9d68062640 -1 osd.5 0 waiting for initial osdmap 2024-03-29T04:09:01.688 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-03-29T04:09:01.709+0000 7f460cade640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.692 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-03-29T04:09:01.712+0000 7f6e3007d640 -1 osd.7 0 waiting for initial osdmap 2024-03-29T04:09:01.692 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-03-29T04:09:01.713+0000 7f9d5fe82640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.694 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-03-29T04:09:01.714+0000 7f4a33ced640 -1 osd.2 0 waiting for initial osdmap 2024-03-29T04:09:01.704 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-03-29T04:09:01.725+0000 7f098e0ce640 -1 osd.4 0 waiting for initial osdmap 2024-03-29T04:09:01.707 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-03-29T04:09:01.728+0000 7f6e27e9d640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.708 INFO:tasks.ceph.osd.3.smithi181.stderr:2024-03-29T04:09:01.729+0000 7f181b929640 -1 osd.3 0 waiting for initial osdmap 2024-03-29T04:09:01.711 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-03-29T04:09:01.732+0000 7f4a2bb0d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.716 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-03-29T04:09:01.737+0000 7f0985eee640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.718 INFO:tasks.ceph.osd.3.smithi181.stderr:2024-03-29T04:09:01.739+0000 7f1813749640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T04:09:01.764 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.783+0000 7f832bf24640 1 Processor -- start 2024-03-29T04:09:01.764 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832bf24640 1 -- start start 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832bf24640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f83240d05f0 0x7f8324070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832bf24640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8324161200 con 0x7f8324160d50 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832bf24640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832406f980 con 0x7f8324070740 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832bf24640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8324160150 con 0x7f83240d05f0 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.784+0000 7f832a4c1640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8324160d50 0x7f83240cccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:40846/0 (socket says 172.21.15.181:40846) 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f832a4c1640 1 -- 172.21.15.181:0/3307596504 learned_addr learned my addr 172.21.15.181:0/3307596504 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8329cc0640 1 --2- 172.21.15.181:0/3307596504 >> v2:172.21.15.187:3300/0 conn(0x7f83240d05f0 0x7f8324070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361198419 0 0) 0x7f8324161200 con 0x7f8324160d50 2024-03-29T04:09:01.767 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8310003520 con 0x7f8324160d50 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3677246872 0 0) 0x7f832406f980 con 0x7f8324070740 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8324161200 con 0x7f8324070740 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8329cc0640 1 -- 172.21.15.181:0/3307596504 >> v1:172.21.15.187:6789/0 conn(0x7f8324070740 legacy=0x7f83240d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8329cc0640 1 -- 172.21.15.181:0/3307596504 >> v1:172.21.15.181:6789/0 conn(0x7f8324160d50 legacy=0x7f83240cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.785+0000 7f8329cc0640 1 -- 172.21.15.181:0/3307596504 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832406fc60 con 0x7f83240d05f0 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f8329cc0640 1 --2- 172.21.15.181:0/3307596504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83240d05f0 0x7f8324070ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8320009d90 tx=0x7f832002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110facdbf3fcdebf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320039070 con 0x7f83240d05f0 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832002cd00 con 0x7f83240d05f0 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f8328cbe640 1 -- 172.21.15.181:0/3307596504 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320034640 con 0x7f83240d05f0 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 -- 172.21.15.181:0/3307596504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83240d05f0 msgr2=0x7f8324070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 --2- 172.21.15.181:0/3307596504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83240d05f0 0x7f8324070ba0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8320009d90 tx=0x7f832002bf50 comp rx=0 tx=0).stop 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 -- 172.21.15.181:0/3307596504 shutdown_connections 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 --2- 172.21.15.181:0/3307596504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83240d05f0 0x7f8324070ba0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 -- 172.21.15.181:0/3307596504 >> 172.21.15.181:0/3307596504 conn(0x7f83240b3e20 msgr2=0x7f83240b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:01.768 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 -- 172.21.15.181:0/3307596504 shutdown_connections 2024-03-29T04:09:01.769 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.786+0000 7f832bf24640 1 -- 172.21.15.181:0/3307596504 wait complete. 2024-03-29T04:09:01.769 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.787+0000 7f832bf24640 1 Processor -- start 2024-03-29T04:09:01.769 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 -- start start 2024-03-29T04:09:01.771 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:01.771 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83240d05f0 0x7f832406cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:01.771 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8324160d50 0x7f832406d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:01.771 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83240711f0 con 0x7f8324070740 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8324070220 con 0x7f8324160d50 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832bf24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f83240cea30 con 0x7f83240d05f0 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f8329cc0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f8329cc0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60934/0 (socket says 172.21.15.181:60934) 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f8329cc0640 1 -- 172.21.15.181:0/3880177080 learned_addr learned my addr 172.21.15.181:0/3880177080 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f83294bf640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83240d05f0 0x7f832406cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.788+0000 7f832a4c1640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8324160d50 0x7f832406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 -- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83240d05f0 msgr2=0x7f832406cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83240d05f0 0x7f832406cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 -- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8324160d50 msgr2=0x7f832406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8324160d50 0x7f832406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 -- 172.21.15.181:0/3880177080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832406db50 con 0x7f8324070740 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f8329cc0640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8320009eb0 tx=0x7f8320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320039050 con 0x7f8324070740 2024-03-29T04:09:01.772 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8320045070 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320009450 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832406de30 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.789+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83240ca220 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.790+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83240758d0 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.790+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 4) ==== 90800+0+0 (secure 0 0 0) 0x7f832003e830 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.791+0000 7f831a7fc640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 0x7f830806e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.791+0000 7f83294bf640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 0x7f830806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.791+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f83200ad4d0 con 0x7f8324070740 2024-03-29T04:09:01.773 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.791+0000 7f83294bf640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 0x7f830806e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83240cc4e0 tx=0x7f8314009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:01.775 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.795+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8320032e20 con 0x7f8324070740 2024-03-29T04:09:01.948 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.969+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f832406c280 con 0x7f8324070740 2024-03-29T04:09:01.949 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.969+0000 7f831a7fc640 1 -- 172.21.15.181:0/3880177080 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f832007ca80 con 0x7f8324070740 2024-03-29T04:09:01.949 INFO:teuthology.misc.health.smithi181.stdout: 2024-03-29T04:09:01.949 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":12,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:08:59.582569+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:01.951 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 msgr2=0x7f830806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.951 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 0x7f830806e960 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83240cc4e0 tx=0x7f8314009040 comp rx=0 tx=0).stop 2024-03-29T04:09:01.951 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 msgr2=0x7f83240c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8320009eb0 tx=0x7f8320004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 shutdown_connections 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8324160d50 0x7f832406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f830806c4a0 0x7f830806e960 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83240d05f0 0x7f832406cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 --2- 172.21.15.181:0/3880177080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8324070740 0x7f83240c85b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 >> 172.21.15.181:0/3880177080 conn(0x7f83240b3e20 msgr2=0x7f83240b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 shutdown_connections 2024-03-29T04:09:01.952 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:01.971+0000 7f832bf24640 1 -- 172.21.15.181:0/3880177080 wait complete. 2024-03-29T04:09:01.960 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T04:09:08.766 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:08.867 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.887+0000 7f79b5500640 1 Processor -- start 2024-03-29T04:09:08.867 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.887+0000 7f79b5500640 1 -- start start 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79b5500640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79b00d0c00 0x7f79b0166150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79b5500640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b015bd00 con 0x7f79b015b780 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79b5500640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b015bfe0 con 0x7f79b0158d60 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79b5500640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79b00729c0 con 0x7f79b00d0c00 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79ae7fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f79b015b780 0x7f79b00d04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41976/0 (socket says 172.21.15.181:41976) 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79ae7fc640 1 -- 172.21.15.181:0/1308002745 learned_addr learned my addr 172.21.15.181:0/1308002745 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79af7fe640 1 --2- 172.21.15.181:0/1308002745 >> v2:172.21.15.187:3300/0 conn(0x7f79b00d0c00 0x7f79b0166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.888+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232861810 0 0) 0x7f79b015bd00 con 0x7f79b015b780 2024-03-29T04:09:08.871 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7998003520 con 0x7f79b015b780 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3677600313 0 0) 0x7f7998003520 con 0x7f79b015b780 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 >> v2:172.21.15.187:3300/0 conn(0x7f79b00d0c00 msgr2=0x7f79b0166150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 --2- 172.21.15.181:0/1308002745 >> v2:172.21.15.187:3300/0 conn(0x7f79b00d0c00 0x7f79b0166150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 >> v1:172.21.15.187:6789/0 conn(0x7f79b0158d60 legacy=0x7f79b015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b015c2c0 con 0x7f79b015b780 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f79a4001670 con 0x7f79b015b780 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 >> v1:172.21.15.181:6789/0 conn(0x7f79b015b780 legacy=0x7f79b00d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 0x7f7998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00729c0 con 0x7f79980039b0 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f799800c160 con 0x7f7998004800 2024-03-29T04:09:08.872 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f799800c360 con 0x7f79980051c0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 0x7f7998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79ae7fc640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79af7fe640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 -- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79980051c0 msgr2=0x7f799800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79980051c0 0x7f799800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 -- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7998004800 msgr2=0x7f7998004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7998004800 0x7f7998004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79af7fe640 1 -- 172.21.15.181:0/1308002745 reap_dead start 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.889+0000 7f79aeffd640 1 -- 172.21.15.181:0/1308002745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7998003520 con 0x7f79980039b0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.890+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 0x7f7998003dd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001e50 tx=0x7f7998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5221065a23cd3edb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.890+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0001a70 con 0x7f79980039b0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.890+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0003b70 con 0x7f79980039b0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.890+0000 7f79adffb640 1 -- 172.21.15.181:0/1308002745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00050d0 con 0x7f79980039b0 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 -- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 msgr2=0x7f7998003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:08.873 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 0x7f7998003dd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001e50 tx=0x7f7998005830 comp rx=0 tx=0).stop 2024-03-29T04:09:08.874 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 -- 172.21.15.181:0/1308002745 shutdown_connections 2024-03-29T04:09:08.874 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 --2- 172.21.15.181:0/1308002745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79980039b0 0x7f7998003dd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.874 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 -- 172.21.15.181:0/1308002745 >> 172.21.15.181:0/1308002745 conn(0x7f79b006b8e0 msgr2=0x7f79b006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:08.874 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 -- 172.21.15.181:0/1308002745 shutdown_connections 2024-03-29T04:09:08.874 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.891+0000 7f79b5500640 1 -- 172.21.15.181:0/1308002745 wait complete. 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.892+0000 7f79b5500640 1 Processor -- start 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.892+0000 7f79b5500640 1 -- start start 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 0x7f79b0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79b0153a60 0x7f79b0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00addc0 con 0x7f79b0153510 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0159140 con 0x7f79980039b0 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79b5500640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79b00d04f0 con 0x7f79b0153a60 2024-03-29T04:09:08.877 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57536/0 (socket says 172.21.15.181:57536) 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 -- 172.21.15.181:0/1403405623 learned_addr learned my addr 172.21.15.181:0/1403405623 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79ae7fc640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79b0153a60 0x7f79b0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 0x7f79b0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 -- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79b0153a60 msgr2=0x7f79b0159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79b0153a60 0x7f79b0159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 -- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 msgr2=0x7f79b0099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 0x7f79b0099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 -- 172.21.15.181:0/1403405623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b0152e10 con 0x7f79b0153510 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 0x7f79b0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.893+0000 7f79af7fe640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f79b00d11d0 tx=0x7f79a8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.894+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8004240 con 0x7f79b0153510 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.894+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a801b070 con 0x7f79b0153510 2024-03-29T04:09:08.878 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.894+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b01530f0 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.894+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a800f710 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.894+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b015de20 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.895+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b00758d0 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.895+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f79a8012040 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.895+0000 7f79977fe640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 0x7f799c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.895+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f79a808b1f0 con 0x7f79b0153510 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.895+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 0x7f799c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.896+0000 7f79aeffd640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 0x7f799c06e9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79a0034000 tx=0x7f79a0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:08.879 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:08.900+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f79a8059ad0 con 0x7f79b0153510 2024-03-29T04:09:09.054 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.074+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79b006f3e0 con 0x7f79b0153510 2024-03-29T04:09:09.054 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.075+0000 7f79977fe640 1 -- 172.21.15.181:0/1403405623 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f79a80593d0 con 0x7f79b0153510 2024-03-29T04:09:09.055 INFO:teuthology.misc.health.smithi181.stdout: 2024-03-29T04:09:09.055 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":16,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:05.611837+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 msgr2=0x7f799c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 0x7f799c06e9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79a0034000 tx=0x7f79a0009030 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 msgr2=0x7f79b0159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f79b00d11d0 tx=0x7f79a8011cf0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 shutdown_connections 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b0153510 0x7f79b0159da0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f799c06c530 0x7f799c06e9f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79b0153a60 0x7f79b0159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 --2- 172.21.15.181:0/1403405623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79980039b0 0x7f79b0099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 >> 172.21.15.181:0/1403405623 conn(0x7f79b006b8e0 msgr2=0x7f79b0157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 shutdown_connections 2024-03-29T04:09:09.057 INFO:teuthology.misc.health.smithi181.stderr:2024-03-29T04:09:09.077+0000 7f79b5500640 1 -- 172.21.15.181:0/1403405623 wait complete. 2024-03-29T04:09:09.066 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T04:09:09.066 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T04:09:09.067 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:09:09.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:09:09.069 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:09:09.069 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:09:09.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:09:09.071 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:09:09.071 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:09:09.071 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:09:09.074 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:09:09.074 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:09:09.074 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:09:09.076 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:09:09.076 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T04:09:09.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:09.145 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2024-03-29T04:09:09.148 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2024-03-29T04:09:09.159 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-03-29T04:09:09.160 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-03-29T04:09:09.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.203+0000 7ff145700640 1 Processor -- start 2024-03-29T04:09:09.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.203+0000 7ff145700640 1 -- start start 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff145700640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400d4a90 0x7ff1400d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff145700640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff1400b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff145700640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1400d36f0 con 0x7ff14015e950 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff145700640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ff1400d3cb0 con 0x7ff1400d4a90 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff145700640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff1400d39d0 con 0x7ff1400d2f00 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400d4a90 0x7ff1400d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400d4a90 0x7ff1400d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57578/0 (socket says 172.21.15.181:57578) 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 -- 172.21.15.181:0/3262589698 learned_addr learned my addr 172.21.15.181:0/3262589698 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13f7fe640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff1400b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 -- 172.21.15.181:0/3262589698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 msgr2=0x7ff1400b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff1400b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 -- 172.21.15.181:0/3262589698 >> v1:172.21.15.187:6789/0 conn(0x7ff14015e950 legacy=0x7ff1400d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13e7fc640 1 -- 172.21.15.181:0/3262589698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1400b7ac0 con 0x7ff1400d4a90 2024-03-29T04:09:09.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.204+0000 7ff13f7fe640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff1400b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:09.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.205+0000 7ff13e7fc640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400d4a90 0x7ff1400d29c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff130009870 tx=0x7ff13002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f488626df9ce8ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.205+0000 7ff13dffb640 1 -- 172.21.15.181:0/3262589698 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13002f840 con 0x7ff1400d4a90 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.205+0000 7ff13dffb640 1 -- 172.21.15.181:0/3262589698 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13002f9e0 con 0x7ff1400d4a90 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.205+0000 7ff13dffb640 1 -- 172.21.15.181:0/3262589698 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13002fdb0 con 0x7ff1400d4a90 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 -- 172.21.15.181:0/3262589698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400d4a90 msgr2=0x7ff1400d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400d4a90 0x7ff1400d29c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff130009870 tx=0x7ff13002ba20 comp rx=0 tx=0).stop 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 -- 172.21.15.181:0/3262589698 shutdown_connections 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff1400b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 --2- 172.21.15.181:0/3262589698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400d4a90 0x7ff1400d29c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 -- 172.21.15.181:0/3262589698 >> 172.21.15.181:0/3262589698 conn(0x7ff1400676e0 msgr2=0x7ff140067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 -- 172.21.15.181:0/3262589698 shutdown_connections 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.206+0000 7ff145700640 1 -- 172.21.15.181:0/3262589698 wait complete. 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.207+0000 7ff145700640 1 Processor -- start 2024-03-29T04:09:09.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.207+0000 7ff145700640 1 -- start start 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff140161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 0x7ff1400aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff14015ed30 con 0x7ff14015e950 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1400d33b0 con 0x7ff1400d4a90 2024-03-29T04:09:09.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff145700640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff14015eeb0 con 0x7ff1400d2f00 2024-03-29T04:09:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57584/0 (socket says 172.21.15.181:57584) 2024-03-29T04:09:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 -- 172.21.15.181:0/3561062403 learned_addr learned my addr 172.21.15.181:0/3561062403 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13e7fc640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 0x7ff1400aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13effd640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff140161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 -- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 msgr2=0x7ff140161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff140161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 -- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 msgr2=0x7ff1400aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 0x7ff1400aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 -- 172.21.15.181:0/3561062403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401608d0 con 0x7ff14015e950 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13e7fc640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 0x7ff1400aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff13f7fe640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff138007ea0 tx=0x7ff138007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.208+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13801a070 con 0x7ff14015e950 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.209+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff140160bb0 con 0x7ff14015e950 2024-03-29T04:09:09.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.209+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff138003bb0 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.209+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13800de70 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.209+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff140161170 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.210+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff140075a10 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.210+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff13801d070 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.210+0000 7ff1277fe640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 0x7ff12c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.210+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff13808b540 con 0x7ff14015e950 2024-03-29T04:09:09.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.210+0000 7ff13effd640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 0x7ff12c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.211+0000 7ff13effd640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 0x7ff12c06e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff134001780 tx=0x7ff134009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.215+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff138010070 con 0x7ff14015e950 2024-03-29T04:09:09.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.392+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff14015df20 con 0x7ff14015e950 2024-03-29T04:09:09.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.393+0000 7ff1277fe640 1 -- 172.21.15.181:0/3561062403 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7ff138059870 con 0x7ff14015e950 2024-03-29T04:09:09.373 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:09.373 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:05.611837+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 msgr2=0x7ff12c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 0x7ff12c06e9f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff134001780 tx=0x7ff134009040 comp rx=0 tx=0).stop 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 msgr2=0x7ff1400b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff138007ea0 tx=0x7ff138007f90 comp rx=0 tx=0).stop 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 shutdown_connections 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff14015e950 0x7ff1400b04b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1400d4a90 0x7ff1400aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff12c06c530 0x7ff12c06e9f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 --2- 172.21.15.181:0/3561062403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1400d2f00 0x7ff140161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 >> 172.21.15.181:0/3561062403 conn(0x7ff1400676e0 msgr2=0x7ff1400d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 shutdown_connections 2024-03-29T04:09:09.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.395+0000 7ff145700640 1 -- 172.21.15.181:0/3561062403 wait complete. 2024-03-29T04:09:09.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:09.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.510+0000 7f5dc2e7e640 1 Processor -- start 2024-03-29T04:09:09.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.510+0000 7f5dc2e7e640 1 -- start start 2024-03-29T04:09:09.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 0x7f5dbc1625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5dbc0d0770 con 0x7f5dbc09aa30 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5dbc15f180 con 0x7f5dbc162120 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc2e7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc15f460 con 0x7f5dbc071020 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57600/0 (socket says 172.21.15.181:57600) 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 -- 172.21.15.181:0/3800572871 learned_addr learned my addr 172.21.15.181:0/3800572871 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc141b640 1 --2- 172.21.15.181:0/3800572871 >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 0x7f5dbc1625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc0c1a640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 -- 172.21.15.181:0/3800572871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 msgr2=0x7f5dbc06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc06c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 -- 172.21.15.181:0/3800572871 >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 msgr2=0x7f5dbc1625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:09.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/3800572871 >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 0x7f5dbc1625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dbbfff640 1 -- 172.21.15.181:0/3800572871 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc15f740 con 0x7f5dbc09aa30 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc141b640 1 --2- 172.21.15.181:0/3800572871 >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 0x7f5dbc1625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.511+0000 7f5dc0c1a640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.512+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06c300 tx=0x7f5da802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6c5bbc364632c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.512+0000 7f5dbb7fe640 1 -- 172.21.15.181:0/3800572871 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8035020 con 0x7f5dbc09aa30 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.512+0000 7f5dbb7fe640 1 -- 172.21.15.181:0/3800572871 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da802d040 con 0x7f5dbc09aa30 2024-03-29T04:09:09.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.512+0000 7f5dbb7fe640 1 -- 172.21.15.181:0/3800572871 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8031a80 con 0x7f5dbc09aa30 2024-03-29T04:09:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/3800572871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 msgr2=0x7f5dbc09aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06c300 tx=0x7f5da802b900 comp rx=0 tx=0).stop 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/3800572871 shutdown_connections 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/3800572871 >> v2:172.21.15.187:3300/0 conn(0x7f5dbc162120 0x7f5dbc1625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc09aeb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/3800572871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/3800572871 >> 172.21.15.181:0/3800572871 conn(0x7f5dbc0b3ce0 msgr2=0x7f5dbc0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/3800572871 shutdown_connections 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.513+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/3800572871 wait complete. 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.514+0000 7f5dc2e7e640 1 Processor -- start 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.514+0000 7f5dc2e7e640 1 -- start start 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc162120 0x7f5dbc153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc06cd80 con 0x7f5dbc09aa30 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc071400 con 0x7f5dbc162120 2024-03-29T04:09:09.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5dbc06f620 con 0x7f5dbc071020 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57616/0 (socket says 172.21.15.181:57616) 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 -- 172.21.15.181:0/1396709599 learned_addr learned my addr 172.21.15.181:0/1396709599 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc141b640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc162120 0x7f5dbc153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc0c1a640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 -- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 msgr2=0x7f5dbc0b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc0b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 -- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc162120 msgr2=0x7f5dbc153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc162120 0x7f5dbc153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 -- 172.21.15.181:0/1396709599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc0af860 con 0x7f5dbc09aa30 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dbbfff640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da8000c00 tx=0x7f5da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da803ac60 con 0x7f5dbc09aa30 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da803ae00 con 0x7f5dbc09aa30 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8044610 con 0x7f5dbc09aa30 2024-03-29T04:09:09.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc0afb40 con 0x7f5dbc09aa30 2024-03-29T04:09:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.515+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc0b0100 con 0x7f5dbc09aa30 2024-03-29T04:09:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.517+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc075890 con 0x7f5dbc09aa30 2024-03-29T04:09:09.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.517+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5da8038020 con 0x7f5dbc09aa30 2024-03-29T04:09:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.517+0000 7f5db97fa640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 0x7f5da006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.517+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5da80ad2a0 con 0x7f5dbc09aa30 2024-03-29T04:09:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.517+0000 7f5dc0c1a640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 0x7f5da006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.518+0000 7f5dc0c1a640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 0x7f5da006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5db0003610 tx=0x7f5db0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.522+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5da803f070 con 0x7f5dbc09aa30 2024-03-29T04:09:09.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.698+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5dbc0d0e80 con 0x7f5dbc09aa30 2024-03-29T04:09:09.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.698+0000 7f5db97fa640 1 -- 172.21.15.181:0/1396709599 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f5da80ad0c0 con 0x7f5dbc09aa30 2024-03-29T04:09:09.678 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:09.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.700+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 msgr2=0x7f5da006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.680 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.700+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 0x7f5da006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5db0003610 tx=0x7f5db0008040 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.700+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 msgr2=0x7f5dbc0b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.700+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5da8000c00 tx=0x7f5da80046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 shutdown_connections 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dbc162120 0x7f5dbc153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dbc09aa30 0x7f5dbc0b2920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5da006c530 0x7f5da006e9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 --2- 172.21.15.181:0/1396709599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5dbc071020 0x7f5dbc0b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 >> 172.21.15.181:0/1396709599 conn(0x7f5dbc0b3ce0 msgr2=0x7f5dbc06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 shutdown_connections 2024-03-29T04:09:09.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.701+0000 7f5dc2e7e640 1 -- 172.21.15.181:0/1396709599 wait complete. 2024-03-29T04:09:09.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.812+0000 7fb306f3d640 1 Processor -- start 2024-03-29T04:09:09.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.812+0000 7fb306f3d640 1 -- start start 2024-03-29T04:09:09.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb30015c400 0x7fb3000cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3000cf3c0 0x7fb3000b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3000b41f0 0x7fb3000cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb3000b4620 con 0x7fb3000cf3c0 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb3000d2850 con 0x7fb30015c400 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb306f3d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3000d2b30 con 0x7fb3000b41f0 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3000cf3c0 0x7fb3000b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3000cf3c0 0x7fb3000b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57618/0 (socket says 172.21.15.181:57618) 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 -- 172.21.15.181:0/166216446 learned_addr learned my addr 172.21.15.181:0/166216446 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb304cd9640 1 --2- 172.21.15.181:0/166216446 >> v2:172.21.15.187:3300/0 conn(0x7fb30015c400 0x7fb3000cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb3054da640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3000b41f0 0x7fb3000cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 -- 172.21.15.181:0/166216446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3000b41f0 msgr2=0x7fb3000cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3000b41f0 0x7fb3000cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 -- 172.21.15.181:0/166216446 >> v2:172.21.15.187:3300/0 conn(0x7fb30015c400 msgr2=0x7fb3000cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 --2- 172.21.15.181:0/166216446 >> v2:172.21.15.187:3300/0 conn(0x7fb30015c400 0x7fb3000cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.813+0000 7fb2fffff640 1 -- 172.21.15.181:0/166216446 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3000d2e10 con 0x7fb3000cf3c0 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb2fffff640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3000b3cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cf270 tx=0x7fb2ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720a191ca1c694f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb2ff7fe640 1 -- 172.21.15.181:0/166216446 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec02e070 con 0x7fb3000cf3c0 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb2ff7fe640 1 -- 172.21.15.181:0/166216446 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ec037070 con 0x7fb3000cf3c0 2024-03-29T04:09:09.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb2ff7fe640 1 -- 172.21.15.181:0/166216446 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec031a00 con 0x7fb3000cf3c0 2024-03-29T04:09:09.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 -- 172.21.15.181:0/166216446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 msgr2=0x7fb3000b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3000b3cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cf270 tx=0x7fb2ec02b880 comp rx=0 tx=0).stop 2024-03-29T04:09:09.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 -- 172.21.15.181:0/166216446 shutdown_connections 2024-03-29T04:09:09.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3000b41f0 0x7fb3000cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 --2- 172.21.15.181:0/166216446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3000b3cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 --2- 172.21.15.181:0/166216446 >> v2:172.21.15.187:3300/0 conn(0x7fb30015c400 0x7fb3000cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.814+0000 7fb306f3d640 1 -- 172.21.15.181:0/166216446 >> 172.21.15.181:0/166216446 conn(0x7fb3000672c0 msgr2=0x7fb3000676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.815+0000 7fb306f3d640 1 -- 172.21.15.181:0/166216446 shutdown_connections 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.815+0000 7fb306f3d640 1 -- 172.21.15.181:0/166216446 wait complete. 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.816+0000 7fb306f3d640 1 Processor -- start 2024-03-29T04:09:09.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.816+0000 7fb306f3d640 1 -- start start 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.816+0000 7fb306f3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3000b41f0 0x7fb300152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.816+0000 7fb306f3d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb30015c400 0x7fb300154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3000d0300 con 0x7fb3000cf3c0 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb30015c7e0 con 0x7fb3000b41f0 2024-03-29T04:09:09.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3000d0480 con 0x7fb30015c400 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57624/0 (socket says 172.21.15.181:57624) 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 -- 172.21.15.181:0/3126557386 learned_addr learned my addr 172.21.15.181:0/3126557386 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb3054da640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb30015c400 0x7fb300154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb304cd9640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3000b41f0 0x7fb300152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.809 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 -- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb30015c400 msgr2=0x7fb300154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb30015c400 0x7fb300154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 -- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3000b41f0 msgr2=0x7fb300152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3000b41f0 0x7fb300152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 -- 172.21.15.181:0/3126557386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb300158410 con 0x7fb3000cf3c0 2024-03-29T04:09:09.810 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fffff640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec02b8c0 tx=0x7fb2ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec032670 con 0x7fb3000cf3c0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ec032830 con 0x7fb3000cf3c0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3001586f0 con 0x7fb3000cf3c0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec032c20 con 0x7fb3000cf3c0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.817+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30015ddf0 con 0x7fb3000cf3c0 2024-03-29T04:09:09.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.818+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb300075a10 con 0x7fb3000cf3c0 2024-03-29T04:09:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.819+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb2ec04c060 con 0x7fb3000cf3c0 2024-03-29T04:09:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.819+0000 7fb2fd7fa640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 0x7fb2e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.819+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb2ec0acc60 con 0x7fb3000cf3c0 2024-03-29T04:09:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.819+0000 7fb304cd9640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 0x7fb2e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:09.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.819+0000 7fb304cd9640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 0x7fb2e406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a30 tx=0x7fb2f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:09.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:09.823+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb2ec040070 con 0x7fb3000cf3c0 2024-03-29T04:09:09.973 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:09.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:10.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.038+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb30015b8c0 con 0x7fb3000cf3c0 2024-03-29T04:09:10.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.039+0000 7fb2fd7fa640 1 -- 172.21.15.181:0/3126557386 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fb2ec07b660 con 0x7fb3000cf3c0 2024-03-29T04:09:10.019 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:10.019 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-29T04:09:10.020 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 2 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 msgr2=0x7fb2e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 0x7fb2e406e9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a30 tx=0x7fb2f4008040 comp rx=0 tx=0).stop 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 msgr2=0x7fb3001f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec02b8c0 tx=0x7fb2ec0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 shutdown_connections 2024-03-29T04:09:10.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb2e406c530 0x7fb2e406e9f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb30015c400 0x7fb300154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3000cf3c0 0x7fb3001f26b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 --2- 172.21.15.181:0/3126557386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3000b41f0 0x7fb300152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 >> 172.21.15.181:0/3126557386 conn(0x7fb3000672c0 msgr2=0x7fb3000b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 shutdown_connections 2024-03-29T04:09:10.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:10.041+0000 7fb306f3d640 1 -- 172.21.15.181:0/3126557386 wait complete. 2024-03-29T04:09:12.033 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:12.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.165+0000 7fd41e172640 1 Processor -- start 2024-03-29T04:09:12.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.165+0000 7fd41e172640 1 -- start start 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd418072230 0x7fd41806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 0x7fd41815f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd41806db10 0x7fd41806def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd418158f10 con 0x7fd418072230 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd41815ef70 con 0x7fd418072610 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd41e172640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd41806c4d0 con 0x7fd41806db10 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd418072230 0x7fd41806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd418072230 0x7fd41806bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57632/0 (socket says 172.21.15.181:57632) 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 -- 172.21.15.181:0/1995239088 learned_addr learned my addr 172.21.15.181:0/1995239088 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd417fff640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd41806db10 0x7fd41806def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 -- 172.21.15.181:0/1995239088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd41806db10 msgr2=0x7fd41806def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd416ffd640 1 --2- 172.21.15.181:0/1995239088 >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 0x7fd41815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd41806db10 0x7fd41806def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 -- 172.21.15.181:0/1995239088 >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 msgr2=0x7fd41815f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 --2- 172.21.15.181:0/1995239088 >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 0x7fd41815f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd4177fe640 1 -- 172.21.15.181:0/1995239088 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4180ce840 con 0x7fd418072230 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.166+0000 7fd416ffd640 1 --2- 172.21.15.181:0/1995239088 >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 0x7fd41815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.167+0000 7fd4177fe640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418072230 0x7fd41806bf90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b02d0 tx=0x7fd40802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7382cfdb546cb25 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.167+0000 7fd4167fc640 1 -- 172.21.15.181:0/1995239088 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035020 con 0x7fd418072230 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.167+0000 7fd4167fc640 1 -- 172.21.15.181:0/1995239088 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd408031870 con 0x7fd418072230 2024-03-29T04:09:12.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.167+0000 7fd4167fc640 1 -- 172.21.15.181:0/1995239088 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408031c40 con 0x7fd418072230 2024-03-29T04:09:12.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 -- 172.21.15.181:0/1995239088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418072230 msgr2=0x7fd41806bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418072230 0x7fd41806bf90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b02d0 tx=0x7fd40802b910 comp rx=0 tx=0).stop 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 -- 172.21.15.181:0/1995239088 shutdown_connections 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd41806db10 0x7fd41806def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 --2- 172.21.15.181:0/1995239088 >> v2:172.21.15.187:3300/0 conn(0x7fd418072610 0x7fd41815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 --2- 172.21.15.181:0/1995239088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd418072230 0x7fd41806bf90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 -- 172.21.15.181:0/1995239088 >> 172.21.15.181:0/1995239088 conn(0x7fd4180d0cb0 msgr2=0x7fd4180d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 -- 172.21.15.181:0/1995239088 shutdown_connections 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.168+0000 7fd41e172640 1 -- 172.21.15.181:0/1995239088 wait complete. 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.169+0000 7fd41e172640 1 Processor -- start 2024-03-29T04:09:12.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.169+0000 7fd41e172640 1 -- start start 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd418072230 0x7fd4180b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd418072610 0x7fd418161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd41815f940 con 0x7fd41806db10 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd41815ec00 con 0x7fd418072230 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd41e172640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd418072ac0 con 0x7fd418072610 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57646/0 (socket says 172.21.15.181:57646) 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 -- 172.21.15.181:0/3077673949 learned_addr learned my addr 172.21.15.181:0/3077673949 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd416ffd640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd418072230 0x7fd4180b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd417fff640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd418072610 0x7fd418161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 -- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd418072610 msgr2=0x7fd418161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd418072610 0x7fd418161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 -- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd418072230 msgr2=0x7fd4180b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd418072230 0x7fd4180b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 -- 172.21.15.181:0/3077673949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181f50d0 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.170+0000 7fd4177fe640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd4080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.171+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408035040 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.171+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd408030d00 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.171+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080456a0 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.171+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4181f4b10 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.171+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4180b36f0 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.172+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd418067360 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.172+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd408038020 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.173+0000 7fd3fffff640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 0x7fd40406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.173+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd4080ad5e0 con 0x7fd41806db10 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.173+0000 7fd416ffd640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 0x7fd40406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.173+0000 7fd416ffd640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 0x7fd40406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd4181f8800 tx=0x7fd40c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.177+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd418067360 con 0x7fd41806db10 2024-03-29T04:09:12.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.393+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd418075a10 con 0x7fd41806db10 2024-03-29T04:09:12.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.393+0000 7fd3fffff640 1 -- 172.21.15.181:0/3077673949 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7fd408080e20 con 0x7fd41806db10 2024-03-29T04:09:12.373 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:12.373 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-29T04:09:12.373 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 3 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 msgr2=0x7fd40406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 0x7fd40406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd4181f8800 tx=0x7fd40c008040 comp rx=0 tx=0).stop 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 msgr2=0x7fd4181f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd4080046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 shutdown_connections 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd40406c530 0x7fd40406e9f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd418072610 0x7fd418161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd418072230 0x7fd4180b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 --2- 172.21.15.181:0/3077673949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41806db10 0x7fd4181f4150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 >> 172.21.15.181:0/3077673949 conn(0x7fd4180d0cb0 msgr2=0x7fd41815d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 shutdown_connections 2024-03-29T04:09:12.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.395+0000 7fd41e172640 1 -- 172.21.15.181:0/3077673949 wait complete. 2024-03-29T04:09:12.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:12.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.507+0000 7f8871874640 1 Processor -- start 2024-03-29T04:09:12.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.507+0000 7f8871874640 1 -- start start 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f8871874640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f886c141530 0x7f886c071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f8871874640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f886c162030 0x7f886c1624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f8871874640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f886c141910 con 0x7f886c0b5240 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f8871874640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f886c071e60 con 0x7f886c162030 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f8871874640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f886c162af0 con 0x7f886c141530 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f886c162030 0x7f886c1624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f886c162030 0x7f886c1624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57648/0 (socket says 172.21.15.181:57648) 2024-03-29T04:09:12.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 -- 172.21.15.181:0/235744523 learned_addr learned my addr 172.21.15.181:0/235744523 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886affd640 1 --2- 172.21.15.181:0/235744523 >> v2:172.21.15.187:3300/0 conn(0x7f886c141530 0x7f886c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 -- 172.21.15.181:0/235744523 >> v2:172.21.15.187:3300/0 conn(0x7f886c141530 msgr2=0x7f886c071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 --2- 172.21.15.181:0/235744523 >> v2:172.21.15.187:3300/0 conn(0x7f886c141530 0x7f886c071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 -- 172.21.15.181:0/235744523 >> v1:172.21.15.187:6789/0 conn(0x7f886c0b5240 legacy=0x7f886c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.508+0000 7f886a7fc640 1 -- 172.21.15.181:0/235744523 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c162dd0 con 0x7f886c162030 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.509+0000 7f886a7fc640 1 --2- 172.21.15.181:0/235744523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c162030 0x7f886c1624e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f886c0681c0 tx=0x7f886002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84a28adb8449d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.509+0000 7f8869ffb640 1 -- 172.21.15.181:0/235744523 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860035020 con 0x7f886c162030 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.509+0000 7f8869ffb640 1 -- 172.21.15.181:0/235744523 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8860031870 con 0x7f886c162030 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.509+0000 7f8869ffb640 1 -- 172.21.15.181:0/235744523 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860031c40 con 0x7f886c162030 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 -- 172.21.15.181:0/235744523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c162030 msgr2=0x7f886c1624e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 --2- 172.21.15.181:0/235744523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c162030 0x7f886c1624e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f886c0681c0 tx=0x7f886002b910 comp rx=0 tx=0).stop 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 -- 172.21.15.181:0/235744523 shutdown_connections 2024-03-29T04:09:12.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 --2- 172.21.15.181:0/235744523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c162030 0x7f886c1624e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 --2- 172.21.15.181:0/235744523 >> v2:172.21.15.187:3300/0 conn(0x7f886c141530 0x7f886c071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 -- 172.21.15.181:0/235744523 >> 172.21.15.181:0/235744523 conn(0x7f886c06f8b0 msgr2=0x7f886c06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 -- 172.21.15.181:0/235744523 shutdown_connections 2024-03-29T04:09:12.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.510+0000 7f8871874640 1 -- 172.21.15.181:0/235744523 wait complete. 2024-03-29T04:09:12.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.511+0000 7f8871874640 1 Processor -- start 2024-03-29T04:09:12.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.511+0000 7f8871874640 1 -- start start 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886c0b5240 0x7f886c13df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f886c162030 0x7f886c1475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f886c0b1950 con 0x7f886c141530 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f886c071400 con 0x7f886c162030 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f8871874640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f886c142320 con 0x7f886c0b5240 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57658/0 (socket says 172.21.15.181:57658) 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 -- 172.21.15.181:0/1457400324 learned_addr learned my addr 172.21.15.181:0/1457400324 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 -- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886c0b5240 msgr2=0x7f886c13df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886c0b5240 0x7f886c13df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 -- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f886c162030 msgr2=0x7f886c1475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886b7fe640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f886c162030 0x7f886c1475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f886c162030 0x7f886c1475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 -- 172.21.15.181:0/1457400324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c13b180 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.512+0000 7f886a7fc640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.513+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860035040 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.513+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8860030d00 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.513+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600456a0 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.513+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c146510 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.513+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c146ad0 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.514+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c075a10 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.514+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8860038020 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.514+0000 7f8853fff640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 0x7f885806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.514+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f88600ad5f0 con 0x7f886c141530 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.514+0000 7f886affd640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 0x7f885806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.515+0000 7f886affd640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 0x7f885806ea20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f885c00e190 tx=0x7f885c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.519+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f886007bf60 con 0x7f886c141530 2024-03-29T04:09:12.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.694+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f886c0d2f00 con 0x7f886c141530 2024-03-29T04:09:12.675 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.695+0000 7f8853fff640 1 -- 172.21.15.181:0/1457400324 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7f886007b860 con 0x7f886c141530 2024-03-29T04:09:12.675 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:12.676 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:05.611837+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 msgr2=0x7f885806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 0x7f885806ea20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f885c00e190 tx=0x7f885c00b040 comp rx=0 tx=0).stop 2024-03-29T04:09:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 msgr2=0x7f886c13ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 shutdown_connections 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f886c162030 0x7f886c1475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f886c141530 0x7f886c13ac40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f885806c560 0x7f885806ea20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 --2- 172.21.15.181:0/1457400324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886c0b5240 0x7f886c13df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 >> 172.21.15.181:0/1457400324 conn(0x7f886c06f8b0 msgr2=0x7f886c06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 shutdown_connections 2024-03-29T04:09:12.679 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.697+0000 7f8871874640 1 -- 172.21.15.181:0/1457400324 wait complete. 2024-03-29T04:09:12.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:09:12.688 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.812+0000 7fc6a4359640 1 Processor -- start 2024-03-29T04:09:12.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.812+0000 7fc6a4359640 1 -- start start 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a4359640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc69c0b3e60 0x7fc69c15f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a4359640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc69c162070 0x7fc69c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a4359640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c162610 con 0x7fc69c0afd60 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a4359640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c06fac0 con 0x7fc69c0b3e60 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a4359640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c1628f0 con 0x7fc69c162070 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc69c0b3e60 0x7fc69c15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc69c0b3e60 0x7fc69c15f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57666/0 (socket says 172.21.15.181:57666) 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 -- 172.21.15.181:0/821079601 learned_addr learned my addr 172.21.15.181:0/821079601 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a28f6640 1 --2- 172.21.15.181:0/821079601 >> v2:172.21.15.187:3300/0 conn(0x7fc69c162070 0x7fc69c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 -- 172.21.15.181:0/821079601 >> v2:172.21.15.187:3300/0 conn(0x7fc69c162070 msgr2=0x7fc69c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:12.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/821079601 >> v2:172.21.15.187:3300/0 conn(0x7fc69c162070 0x7fc69c06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 -- 172.21.15.181:0/821079601 >> v1:172.21.15.187:6789/0 conn(0x7fc69c0afd60 legacy=0x7fc69c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.813+0000 7fc6a18f4640 1 -- 172.21.15.181:0/821079601 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c162bd0 con 0x7fc69c0b3e60 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.814+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/821079601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b3e60 0x7fc69c15f500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69c155b70 tx=0x7fc68c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8113537e3403a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.814+0000 7fc6a10f3640 1 -- 172.21.15.181:0/821079601 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c034070 con 0x7fc69c0b3e60 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.814+0000 7fc6a10f3640 1 -- 172.21.15.181:0/821079601 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c030860 con 0x7fc69c0b3e60 2024-03-29T04:09:12.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.814+0000 7fc6a10f3640 1 -- 172.21.15.181:0/821079601 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c030c30 con 0x7fc69c0b3e60 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 -- 172.21.15.181:0/821079601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b3e60 msgr2=0x7fc69c15f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 --2- 172.21.15.181:0/821079601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b3e60 0x7fc69c15f500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69c155b70 tx=0x7fc68c02b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 -- 172.21.15.181:0/821079601 shutdown_connections 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 --2- 172.21.15.181:0/821079601 >> v2:172.21.15.187:3300/0 conn(0x7fc69c162070 0x7fc69c06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 --2- 172.21.15.181:0/821079601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b3e60 0x7fc69c15f500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 -- 172.21.15.181:0/821079601 >> 172.21.15.181:0/821079601 conn(0x7fc69c0d0840 msgr2=0x7fc69c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 -- 172.21.15.181:0/821079601 shutdown_connections 2024-03-29T04:09:12.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.815+0000 7fc6a4359640 1 -- 172.21.15.181:0/821079601 wait complete. 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.816+0000 7fc6a4359640 1 Processor -- start 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.816+0000 7fc6a4359640 1 -- start start 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 0x7fc69c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c162070 0x7fc69c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c06c040 con 0x7fc69c0afd60 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c162eb0 con 0x7fc69c162070 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c06c1c0 con 0x7fc69c0b3e60 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57670/0 (socket says 172.21.15.181:57670) 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 -- 172.21.15.181:0/2420846501 learned_addr learned my addr 172.21.15.181:0/2420846501 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:12.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 0x7fc69c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a28f6640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c162070 0x7fc69c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 -- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 msgr2=0x7fc69c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 0x7fc69c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 -- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c162070 msgr2=0x7fc69c157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c162070 0x7fc69c157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 -- 172.21.15.181:0/2420846501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c1f78c0 con 0x7fc69c0afd60 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 0x7fc69c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a20f5640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc69400c790 tx=0x7fc69400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694016020 con 0x7fc69c0afd60 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc694006d50 con 0x7fc69c0afd60 2024-03-29T04:09:12.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c1547e0 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694003bb0 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.817+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1544d0 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.818+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c1f7a90 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.819+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc69400a850 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.819+0000 7fc692ffd640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.819+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.819+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc69408af60 con 0x7fc69c0afd60 2024-03-29T04:09:12.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.819+0000 7fc6a18f4640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 0x7fc68006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc68c036000 tx=0x7fc68c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:12.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:12.823+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc69c1f7a90 con 0x7fc69c0afd60 2024-03-29T04:09:12.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.001+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc69c075890 con 0x7fc69c0afd60 2024-03-29T04:09:12.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.001+0000 7fc692ffd640 1 -- 172.21.15.181:0/2420846501 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fc694020020 con 0x7fc69c0afd60 2024-03-29T04:09:12.981 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:12.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 msgr2=0x7fc68006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 0x7fc68006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc68c036000 tx=0x7fc68c03b040 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 msgr2=0x7fc69c1f8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc69400c790 tx=0x7fc69400cc50 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 shutdown_connections 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69c162070 0x7fc69c157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc69c0b3e60 0x7fc69c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 --2- 172.21.15.181:0/2420846501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0afd60 0x7fc69c1f8360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 >> 172.21.15.181:0/2420846501 conn(0x7fc69c0d0840 msgr2=0x7fc69c06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 shutdown_connections 2024-03-29T04:09:12.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.003+0000 7fc6a4359640 1 -- 172.21.15.181:0/2420846501 wait complete. 2024-03-29T04:09:12.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:09:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.113+0000 7f6b36dca640 1 Processor -- start 2024-03-29T04:09:13.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 -- start start 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b300d4a90 0x7f6b300d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b300d2f00 0x7f6b300b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b300d36f0 con 0x7f6b3015e950 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f6b300d3cb0 con 0x7f6b300d4a90 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b36dca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b300d39d0 con 0x7f6b300d2f00 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b2ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b300d4a90 0x7f6b300d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.114+0000 7f6b2ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b300d4a90 0x7f6b300d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57674/0 (socket says 172.21.15.181:57674) 2024-03-29T04:09:13.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 -- 172.21.15.181:0/2180053944 learned_addr learned my addr 172.21.15.181:0/2180053944 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b35367640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b300d2f00 0x7f6b300b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 -- 172.21.15.181:0/2180053944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b300d2f00 msgr2=0x7f6b300b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b300d2f00 0x7f6b300b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 -- 172.21.15.181:0/2180053944 >> v1:172.21.15.187:6789/0 conn(0x7f6b3015e950 legacy=0x7f6b300d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 -- 172.21.15.181:0/2180053944 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300b7ac0 con 0x7f6b300d4a90 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.115+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d4a90 0x7f6b300d29c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009870 tx=0x7f6b2402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78496acf56e2d21c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b2f7fe640 1 -- 172.21.15.181:0/2180053944 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24034070 con 0x7f6b300d4a90 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b2f7fe640 1 -- 172.21.15.181:0/2180053944 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24030840 con 0x7f6b300d4a90 2024-03-29T04:09:13.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b2f7fe640 1 -- 172.21.15.181:0/2180053944 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24030c10 con 0x7f6b300d4a90 2024-03-29T04:09:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 -- 172.21.15.181:0/2180053944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d4a90 msgr2=0x7f6b300d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d4a90 0x7f6b300d29c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009870 tx=0x7f6b2402b980 comp rx=0 tx=0).stop 2024-03-29T04:09:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 -- 172.21.15.181:0/2180053944 shutdown_connections 2024-03-29T04:09:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b300d2f00 0x7f6b300b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 --2- 172.21.15.181:0/2180053944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d4a90 0x7f6b300d29c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 -- 172.21.15.181:0/2180053944 >> 172.21.15.181:0/2180053944 conn(0x7f6b300676e0 msgr2=0x7f6b30067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 -- 172.21.15.181:0/2180053944 shutdown_connections 2024-03-29T04:09:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.116+0000 7f6b36dca640 1 -- 172.21.15.181:0/2180053944 wait complete. 2024-03-29T04:09:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.117+0000 7f6b36dca640 1 Processor -- start 2024-03-29T04:09:13.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.117+0000 7f6b36dca640 1 -- start start 2024-03-29T04:09:13.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 0x7f6b300aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 0x7f6b301f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3015ed30 con 0x7f6b300d2f00 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300d33b0 con 0x7f6b300d4a90 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b36dca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3015eeb0 con 0x7f6b3015e950 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b34b66640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b34b66640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57684/0 (socket says 172.21.15.181:57684) 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b34b66640 1 -- 172.21.15.181:0/911806731 learned_addr learned my addr 172.21.15.181:0/911806731 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b35367640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 0x7f6b301f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.118+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 0x7f6b300aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 -- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 msgr2=0x7f6b301f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 0x7f6b301f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 -- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 msgr2=0x7f6b300aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 0x7f6b300aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 -- 172.21.15.181:0/911806731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b301f7b90 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 0x7f6b300aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b35367640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 0x7f6b301f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b34b66640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c430 tx=0x7f6b2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20011790 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20017070 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b301e8f10 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20011b70 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.119+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b301e94d0 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.120+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b300cb050 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.120+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b20011d10 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.121+0000 7f6b2d7fa640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 0x7f6b1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.121+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6b2008b220 con 0x7f6b300d2f00 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.121+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 0x7f6b1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.121+0000 7f6b2ffff640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 0x7f6b1c06e9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b24036000 tx=0x7f6b2403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.125+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6b20059b90 con 0x7f6b300d2f00 2024-03-29T04:09:13.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.303+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6b30075a10 con 0x7f6b300d2f00 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.303+0000 7f6b2d7fa640 1 -- 172.21.15.181:0/911806731 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f6b20059490 con 0x7f6b300d2f00 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:09:13.283 INFO:teuthology.orchestra.run.smithi181.stdout:] 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 msgr2=0x7f6b1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 0x7f6b1c06e9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b24036000 tx=0x7f6b2403b040 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 msgr2=0x7f6b301f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c430 tx=0x7f6b2000c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 shutdown_connections 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6b1c06c530 0x7f6b1c06e9f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b3015e950 0x7f6b301f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b300d4a90 0x7f6b300aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 --2- 172.21.15.181:0/911806731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b300d2f00 0x7f6b301f7650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 >> 172.21.15.181:0/911806731 conn(0x7f6b300676e0 msgr2=0x7f6b300d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 shutdown_connections 2024-03-29T04:09:13.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.305+0000 7f6b36dca640 1 -- 172.21.15.181:0/911806731 wait complete. 2024-03-29T04:09:13.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:09:13.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:09:13.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.420+0000 7f31a9ceb640 1 Processor -- start 2024-03-29T04:09:13.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.420+0000 7f31a9ceb640 1 -- start start 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a4145ff0 0x7f31a406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a415ca90 0x7f31a40cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31a40cde10 0x7f31a40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f31a40d1fc0 con 0x7f31a415ca90 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f31a40d0eb0 con 0x7f31a40cde10 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f31a9ceb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31a40d0610 con 0x7f31a4145ff0 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f319affd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a415ca90 0x7f31a40cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f319affd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a415ca90 0x7f31a40cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57696/0 (socket says 172.21.15.181:57696) 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.421+0000 7f319affd640 1 -- 172.21.15.181:0/3216877349 learned_addr learned my addr 172.21.15.181:0/3216877349 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 -- 172.21.15.181:0/3216877349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a4145ff0 msgr2=0x7f31a406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f31a37fe640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a4145ff0 0x7f31a406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f31a3fff640 1 --2- 172.21.15.181:0/3216877349 >> v2:172.21.15.187:3300/0 conn(0x7f31a40cde10 0x7f31a40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a4145ff0 0x7f31a406bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 -- 172.21.15.181:0/3216877349 >> v2:172.21.15.187:3300/0 conn(0x7f31a40cde10 msgr2=0x7f31a40ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:13.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 --2- 172.21.15.181:0/3216877349 >> v2:172.21.15.187:3300/0 conn(0x7f31a40cde10 0x7f31a40ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 -- 172.21.15.181:0/3216877349 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a40d08f0 con 0x7f31a415ca90 2024-03-29T04:09:13.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.422+0000 7f319affd640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a415ca90 0x7f31a40cfec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f318c009870 tx=0x7f318c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52d25881944cc0e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a2ffd640 1 -- 172.21.15.181:0/3216877349 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0317c0 con 0x7f31a415ca90 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a2ffd640 1 -- 172.21.15.181:0/3216877349 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c037070 con 0x7f31a415ca90 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a2ffd640 1 -- 172.21.15.181:0/3216877349 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c031b80 con 0x7f31a415ca90 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3216877349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a415ca90 msgr2=0x7f31a40cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a415ca90 0x7f31a40cfec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f318c009870 tx=0x7f318c02b860 comp rx=0 tx=0).stop 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3216877349 shutdown_connections 2024-03-29T04:09:13.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3216877349 >> v2:172.21.15.187:3300/0 conn(0x7f31a40cde10 0x7f31a40ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a415ca90 0x7f31a40cfec0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3216877349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a4145ff0 0x7f31a406bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3216877349 >> 172.21.15.181:0/3216877349 conn(0x7f31a40d1180 msgr2=0x7f31a40d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3216877349 shutdown_connections 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.423+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3216877349 wait complete. 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.424+0000 7f31a9ceb640 1 Processor -- start 2024-03-29T04:09:13.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.424+0000 7f31a9ceb640 1 -- start start 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31a40cde10 0x7f31a415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a415ca90 0x7f31a413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40724e0 con 0x7f31a4145ff0 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31a406fab0 con 0x7f31a40cde10 2024-03-29T04:09:13.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f31a9ceb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31a4072660 con 0x7f31a415ca90 2024-03-29T04:09:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f319affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f319affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57708/0 (socket says 172.21.15.181:57708) 2024-03-29T04:09:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.425+0000 7f319affd640 1 -- 172.21.15.181:0/3143512541 learned_addr learned my addr 172.21.15.181:0/3143512541 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a37fe640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31a40cde10 0x7f31a415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a3fff640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a415ca90 0x7f31a413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 -- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a415ca90 msgr2=0x7f31a413ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a415ca90 0x7f31a413ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 -- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31a40cde10 msgr2=0x7f31a415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31a40cde10 0x7f31a415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 -- 172.21.15.181:0/3143512541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a41f7870 con 0x7f31a4145ff0 2024-03-29T04:09:13.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f319affd640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c032860 con 0x7f31a4145ff0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c032a00 con 0x7f31a4145ff0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a41f7b50 con 0x7f31a4145ff0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c032df0 con 0x7f31a4145ff0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.426+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41f8110 con 0x7f31a4145ff0 2024-03-29T04:09:13.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.427+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a4068780 con 0x7f31a4145ff0 2024-03-29T04:09:13.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.427+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f318c04c060 con 0x7f31a4145ff0 2024-03-29T04:09:13.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.428+0000 7f31a0ff9640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 0x7f318406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:13.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.428+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f318c0acd90 con 0x7f31a4145ff0 2024-03-29T04:09:13.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.428+0000 7f31a37fe640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 0x7f318406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.428+0000 7f31a37fe640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 0x7f318406e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:13.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.432+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31a4068780 con 0x7f31a4145ff0 2024-03-29T04:09:13.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:13.609+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f31a40ce860 con 0x7f31a4145ff0 2024-03-29T04:09:14.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.278+0000 7f31a0ff9640 1 -- 172.21.15.181:0/3143512541 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f31a40ce860 con 0x7f31a4145ff0 2024-03-29T04:09:14.258 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 msgr2=0x7f318406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 0x7f318406e9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194008040 comp rx=0 tx=0).stop 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 msgr2=0x7f31a413f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 shutdown_connections 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f318406c530 0x7f318406e9f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31a415ca90 0x7f31a413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a4145ff0 0x7f31a413f9d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 --2- 172.21.15.181:0/3143512541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31a40cde10 0x7f31a415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 >> 172.21.15.181:0/3143512541 conn(0x7f31a40d1180 msgr2=0x7f31a41408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:14.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.281+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 shutdown_connections 2024-03-29T04:09:14.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.282+0000 7f31a9ceb640 1 -- 172.21.15.181:0/3143512541 wait complete. 2024-03-29T04:09:14.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-29T04:09:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 Processor -- start 2024-03-29T04:09:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 -- start start 2024-03-29T04:09:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d6815e4f0 0x7f4d680b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4d680d3ee0 0x7f4d680d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f4d680d3a50 con 0x7f4d6815e4f0 2024-03-29T04:09:14.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4d680d2e40 con 0x7f4d680d3ee0 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.411+0000 7f4d6cfbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4d680d3120 con 0x7f4d680b5e10 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d6815e4f0 0x7f4d680b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d6815e4f0 0x7f4d680b9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57714/0 (socket says 172.21.15.181:57714) 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 -- 172.21.15.181:0/101585981 learned_addr learned my addr 172.21.15.181:0/101585981 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d5ffff640 1 --2- 172.21.15.181:0/101585981 >> v2:172.21.15.187:3300/0 conn(0x7f4d680d3ee0 0x7f4d680d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d66d9d640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 -- 172.21.15.181:0/101585981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 msgr2=0x7f4d680b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 -- 172.21.15.181:0/101585981 >> v2:172.21.15.187:3300/0 conn(0x7f4d680d3ee0 msgr2=0x7f4d680d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 --2- 172.21.15.181:0/101585981 >> v2:172.21.15.187:3300/0 conn(0x7f4d680d3ee0 0x7f4d680d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.412+0000 7f4d6659c640 1 -- 172.21.15.181:0/101585981 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d680d3400 con 0x7f4d6815e4f0 2024-03-29T04:09:14.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6659c640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d680b9f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d680d0ce0 tx=0x7f4d5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b74c8b6b2790e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d65d9b640 1 -- 172.21.15.181:0/101585981 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58034070 con 0x7f4d6815e4f0 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d65d9b640 1 -- 172.21.15.181:0/101585981 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d58030860 con 0x7f4d6815e4f0 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d65d9b640 1 -- 172.21.15.181:0/101585981 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58030c30 con 0x7f4d6815e4f0 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/101585981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 msgr2=0x7f4d680b9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d680b9f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d680d0ce0 tx=0x7f4d5802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/101585981 shutdown_connections 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/101585981 >> v2:172.21.15.187:3300/0 conn(0x7f4d680d3ee0 0x7f4d680d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/101585981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d680b9f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/101585981 >> 172.21.15.181:0/101585981 conn(0x7f4d680678f0 msgr2=0x7f4d680d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/101585981 shutdown_connections 2024-03-29T04:09:14.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.413+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/101585981 wait complete. 2024-03-29T04:09:14.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 Processor -- start 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 -- start start 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 0x7f4d6815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6806c580 con 0x7f4d6815e4f0 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6815e8d0 con 0x7f4d680d3ee0 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d6cfbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6806c700 con 0x7f4d680b5e10 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d66d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d66d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57728/0 (socket says 172.21.15.181:57728) 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.415+0000 7f4d66d9d640 1 -- 172.21.15.181:0/2894701237 learned_addr learned my addr 172.21.15.181:0/2894701237 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d5ffff640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 0x7f4d6815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d6659c640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 -- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 msgr2=0x7f4d680cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 -- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 msgr2=0x7f4d6815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 0x7f4d6815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 -- 172.21.15.181:0/2894701237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d680cb500 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d5ffff640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 0x7f4d6815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d66d9d640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009dd0 tx=0x7f4d6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60018020 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d60014480 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60014870 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d680cb7e0 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.416+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d680cbda0 con 0x7f4d6815e4f0 2024-03-29T04:09:14.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.417+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d68075890 con 0x7f4d6815e4f0 2024-03-29T04:09:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.417+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d60005d10 con 0x7f4d6815e4f0 2024-03-29T04:09:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.418+0000 7f4d5effd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 0x7f4d4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.418+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f4d6008bfd0 con 0x7f4d6815e4f0 2024-03-29T04:09:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.418+0000 7f4d6659c640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 0x7f4d4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:14.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.418+0000 7f4d6659c640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 0x7f4d4806e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d58036000 tx=0x7f4d5803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.423+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4d60022070 con 0x7f4d6815e4f0 2024-03-29T04:09:14.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:14.599+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f4d681619d0 con 0x7f4d6815e4f0 2024-03-29T04:09:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.280+0000 7f4d5effd640 1 -- 172.21.15.181:0/2894701237 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f4d60010040 con 0x7f4d6815e4f0 2024-03-29T04:09:15.260 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 msgr2=0x7f4d4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 0x7f4d4806e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d58036000 tx=0x7f4d5803b040 comp rx=0 tx=0).stop 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 msgr2=0x7f4d6814cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009dd0 tx=0x7f4d6000c490 comp rx=0 tx=0).stop 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 shutdown_connections 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d6815e4f0 0x7f4d6814cad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d680d3ee0 0x7f4d6815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4d4806c530 0x7f4d4806e9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 --2- 172.21.15.181:0/2894701237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d680b5e10 0x7f4d680cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 >> 172.21.15.181:0/2894701237 conn(0x7f4d680678f0 msgr2=0x7f4d6815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 shutdown_connections 2024-03-29T04:09:15.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.283+0000 7f4d6cfbd640 1 -- 172.21.15.181:0/2894701237 wait complete. 2024-03-29T04:09:15.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:09:15.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.412+0000 7fe2474a7640 1 Processor -- start 2024-03-29T04:09:15.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.412+0000 7fe2474a7640 1 -- start start 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe2474a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 0x7fe240072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe2474a7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe240070180 con 0x7fe240145dc0 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe2474a7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe240070460 con 0x7fe2400733e0 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe2474a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe240070740 con 0x7fe24006b580 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe245243640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe240145dc0 0x7fe24006f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:41994/0 (socket says 172.21.15.181:41994) 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe245243640 1 -- 172.21.15.181:0/49213234 learned_addr learned my addr 172.21.15.181:0/49213234 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:15.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe244a42640 1 --2- 172.21.15.181:0/49213234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 0x7fe240072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506094813 0 0) 0x7fe240070180 con 0x7fe240145dc0 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe22c003520 con 0x7fe240145dc0 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300808114 0 0) 0x7fe240070460 con 0x7fe2400733e0 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe240070180 con 0x7fe2400733e0 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe244a42640 1 -- 172.21.15.181:0/49213234 >> v1:172.21.15.187:6789/0 conn(0x7fe2400733e0 legacy=0x7fe2400cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe244a42640 1 -- 172.21.15.181:0/49213234 >> v1:172.21.15.181:6789/0 conn(0x7fe240145dc0 legacy=0x7fe24006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.413+0000 7fe244a42640 1 -- 172.21.15.181:0/49213234 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe240070a20 con 0x7fe24006b580 2024-03-29T04:09:15.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.414+0000 7fe244a42640 1 --2- 172.21.15.181:0/49213234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 0x7fe240072ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097d0 tx=0x7fe23002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce92e180dfde9d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:15.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.414+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230035020 con 0x7fe24006b580 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.414+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe230031890 con 0x7fe24006b580 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.414+0000 7fe237fff640 1 -- 172.21.15.181:0/49213234 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230031c40 con 0x7fe24006b580 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 -- 172.21.15.181:0/49213234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 msgr2=0x7fe240072ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 --2- 172.21.15.181:0/49213234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 0x7fe240072ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2300097d0 tx=0x7fe23002b930 comp rx=0 tx=0).stop 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 -- 172.21.15.181:0/49213234 shutdown_connections 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 --2- 172.21.15.181:0/49213234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe24006b580 0x7fe240072ea0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 -- 172.21.15.181:0/49213234 >> 172.21.15.181:0/49213234 conn(0x7fe24006e720 msgr2=0x7fe24006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 -- 172.21.15.181:0/49213234 shutdown_connections 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.415+0000 7fe2474a7640 1 -- 172.21.15.181:0/49213234 wait complete. 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.416+0000 7fe2474a7640 1 Processor -- start 2024-03-29T04:09:15.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.416+0000 7fe2474a7640 1 -- start start 2024-03-29T04:09:15.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:15.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400733e0 0x7fe2400b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:15.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe240145dc0 0x7fe2400b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:15.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d0770 con 0x7fe24006b580 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d05f0 con 0x7fe2400733e0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400d08f0 con 0x7fe240145dc0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57732/0 (socket says 172.21.15.181:57732) 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 -- 172.21.15.181:0/506285467 learned_addr learned my addr 172.21.15.181:0/506285467 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe244a42640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400733e0 0x7fe2400b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245a44640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe240145dc0 0x7fe2400b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 -- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe240145dc0 msgr2=0x7fe2400b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe240145dc0 0x7fe2400b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 -- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400733e0 msgr2=0x7fe2400b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400733e0 0x7fe2400b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 -- 172.21.15.181:0/506285467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24013ada0 con 0x7fe24006b580 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe245243640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c440 tx=0x7fe23c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:15.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017070 con 0x7fe24006b580 2024-03-29T04:09:15.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe23c005ce0 con 0x7fe24006b580 2024-03-29T04:09:15.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe240138d70 con 0x7fe24006b580 2024-03-29T04:09:15.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.417+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c007270 con 0x7fe24006b580 2024-03-29T04:09:15.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.418+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe240139330 con 0x7fe24006b580 2024-03-29T04:09:15.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.419+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2400687f0 con 0x7fe24006b580 2024-03-29T04:09:15.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.419+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe23c01a070 con 0x7fe24006b580 2024-03-29T04:09:15.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.420+0000 7fe235ffb640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 0x7fe22406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:15.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.420+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fe23c08a970 con 0x7fe24006b580 2024-03-29T04:09:15.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.420+0000 7fe244a42640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 0x7fe22406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:15.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.420+0000 7fe244a42640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 0x7fe22406e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe230036000 tx=0x7fe230008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:15.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.424+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe2400687f0 con 0x7fe24006b580 2024-03-29T04:09:15.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:15.638+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe240075890 con 0x7fe24006b580 2024-03-29T04:09:16.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.301+0000 7fe235ffb640 1 -- 172.21.15.181:0/506285467 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe23c058920 con 0x7fe24006b580 2024-03-29T04:09:16.281 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T04:09:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.308+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 msgr2=0x7fe22406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 0x7fe22406e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe230036000 tx=0x7fe230008030 comp rx=0 tx=0).stop 2024-03-29T04:09:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 msgr2=0x7fe24015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c440 tx=0x7fe23c00c900 comp rx=0 tx=0).stop 2024-03-29T04:09:16.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 shutdown_connections 2024-03-29T04:09:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe22406c530 0x7fe22406e9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe240145dc0 0x7fe2400b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2400733e0 0x7fe2400b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 --2- 172.21.15.181:0/506285467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe24006b580 0x7fe24015bab0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c440 tx=0x7fe23c00c900 comp rx=0 tx=0).stop 2024-03-29T04:09:16.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.309+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 >> 172.21.15.181:0/506285467 conn(0x7fe24006e720 msgr2=0x7fe2400cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:16.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.311+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 shutdown_connections 2024-03-29T04:09:16.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.312+0000 7fe2474a7640 1 -- 172.21.15.181:0/506285467 wait complete. 2024-03-29T04:09:16.313 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-29T04:09:16.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-29T04:09:16.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.442+0000 7fbe9a5b5640 1 Processor -- start 2024-03-29T04:09:16.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.442+0000 7fbe9a5b5640 1 -- start start 2024-03-29T04:09:16.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe9a5b5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe940d05b0 0x7fbe941660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:16.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe9a5b5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940b2fc0 con 0x7fbe94144a80 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe9a5b5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940b32a0 con 0x7fbe940cd950 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe9a5b5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbe940b3580 con 0x7fbe940d05b0 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe98b52640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe940d05b0 0x7fbe941660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe98b52640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe940d05b0 0x7fbe941660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57740/0 (socket says 172.21.15.181:57740) 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe93fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe94144a80 0x7fbe94071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42000/0 (socket says 172.21.15.181:42000) 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.443+0000 7fbe98b52640 1 -- 172.21.15.181:0/1344158126 learned_addr learned my addr 172.21.15.181:0/1344158126 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe98b52640 1 -- 172.21.15.181:0/1344158126 >> v1:172.21.15.187:6789/0 conn(0x7fbe940cd950 legacy=0x7fbe940cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe98b52640 1 -- 172.21.15.181:0/1344158126 >> v1:172.21.15.181:6789/0 conn(0x7fbe94144a80 legacy=0x7fbe94071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe98b52640 1 -- 172.21.15.181:0/1344158126 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940b3860 con 0x7fbe940d05b0 2024-03-29T04:09:16.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe98b52640 1 --2- 172.21.15.181:0/1344158126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940d05b0 0x7fbe941660c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461089a9b73ad5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:16.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe92ffd640 1 -- 172.21.15.181:0/1344158126 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c02cbb0 con 0x7fbe940d05b0 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe92ffd640 1 -- 172.21.15.181:0/1344158126 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c02cd50 con 0x7fbe940d05b0 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.444+0000 7fbe92ffd640 1 -- 172.21.15.181:0/1344158126 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c034610 con 0x7fbe940d05b0 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/1344158126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940d05b0 msgr2=0x7fbe941660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/1344158126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940d05b0 0x7fbe941660c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c02c010 comp rx=0 tx=0).stop 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/1344158126 shutdown_connections 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/1344158126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940d05b0 0x7fbe941660c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/1344158126 >> 172.21.15.181:0/1344158126 conn(0x7fbe9406f060 msgr2=0x7fbe9406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/1344158126 shutdown_connections 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.445+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/1344158126 wait complete. 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.446+0000 7fbe9a5b5640 1 Processor -- start 2024-03-29T04:09:16.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.446+0000 7fbe9a5b5640 1 -- start start 2024-03-29T04:09:16.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:16.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe940d05b0 0x7fbe9413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:16.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe94144a80 0x7fbe9406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:16.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94144e60 con 0x7fbe940cd950 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe94072280 con 0x7fbe940d05b0 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe9a5b5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbe94071620 con 0x7fbe94144a80 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57750/0 (socket says 172.21.15.181:57750) 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 -- 172.21.15.181:0/3942429421 learned_addr learned my addr 172.21.15.181:0/3942429421 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe98b52640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe94144a80 0x7fbe9406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe937fe640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe940d05b0 0x7fbe9413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 -- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe94144a80 msgr2=0x7fbe9406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe94144a80 0x7fbe9406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 -- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe940d05b0 msgr2=0x7fbe9413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:16.430 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe940d05b0 0x7fbe9413dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.447+0000 7fbe93fff640 1 -- 172.21.15.181:0/3942429421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe94138b50 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe93fff640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c410 tx=0x7fbe8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840074e0 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe84007680 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe94138e30 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840049e0 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.448+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941393f0 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.449+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe94075900 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.449+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe8401a020 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.450+0000 7fbe90ff9640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 0x7fbe8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.450+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbe8408acc0 con 0x7fbe940cd950 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.450+0000 7fbe937fe640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 0x7fbe8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:16.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.450+0000 7fbe937fe640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 0x7fbe8006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9415ae80 tx=0x7fbe88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:16.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.454+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe84058d20 con 0x7fbe940cd950 2024-03-29T04:09:16.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:16.637+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fbe940726f0 con 0x7fbe940cd950 2024-03-29T04:09:17.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.299+0000 7fbe90ff9640 1 -- 172.21.15.181:0/3942429421 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fbe8405f070 con 0x7fbe940cd950 2024-03-29T04:09:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 msgr2=0x7fbe8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 0x7fbe8006e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9415ae80 tx=0x7fbe88009040 comp rx=0 tx=0).stop 2024-03-29T04:09:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 msgr2=0x7fbe94138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c410 tx=0x7fbe8400c8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 shutdown_connections 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8006c530 0x7fbe8006e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe94144a80 0x7fbe9406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe940d05b0 0x7fbe9413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 --2- 172.21.15.181:0/3942429421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe940cd950 0x7fbe94138240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.302+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 >> 172.21.15.181:0/3942429421 conn(0x7fbe9406f060 msgr2=0x7fbe94071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.303+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 shutdown_connections 2024-03-29T04:09:17.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.303+0000 7fbe9a5b5640 1 -- 172.21.15.181:0/3942429421 wait complete. 2024-03-29T04:09:17.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:09:17.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 Processor -- start 2024-03-29T04:09:17.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 -- start start 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198815c260 0x7f198815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19880b4300 0x7f198815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1988071110 con 0x7f19880cc7c0 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f19880713f0 con 0x7f198815c260 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f198ea14640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19880716d0 con 0x7f19880b4300 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f1987fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198815c260 0x7f198815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.428+0000 7f1987fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f198815c260 0x7f198815fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57756/0 (socket says 172.21.15.181:57756) 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 -- 172.21.15.181:0/2374241207 learned_addr learned my addr 172.21.15.181:0/2374241207 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f198cfb1640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19880b4300 0x7f198815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 -- 172.21.15.181:0/2374241207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19880b4300 msgr2=0x7f198815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19880b4300 0x7f198815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 -- 172.21.15.181:0/2374241207 >> v1:172.21.15.187:6789/0 conn(0x7f19880cc7c0 legacy=0x7f19880b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 -- 172.21.15.181:0/2374241207 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19880719b0 con 0x7f198815c260 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.429+0000 7f1987fff640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198815c260 0x7f198815fe10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1978009870 tx=0x7f197802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200359c277e6dc36 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f1986ffd640 1 -- 172.21.15.181:0/2374241207 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978034070 con 0x7f198815c260 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f1986ffd640 1 -- 172.21.15.181:0/2374241207 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1978030840 con 0x7f198815c260 2024-03-29T04:09:17.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f1986ffd640 1 -- 172.21.15.181:0/2374241207 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978030c10 con 0x7f198815c260 2024-03-29T04:09:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 -- 172.21.15.181:0/2374241207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198815c260 msgr2=0x7f198815fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198815c260 0x7f198815fe10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1978009870 tx=0x7f197802b980 comp rx=0 tx=0).stop 2024-03-29T04:09:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 -- 172.21.15.181:0/2374241207 shutdown_connections 2024-03-29T04:09:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19880b4300 0x7f198815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 --2- 172.21.15.181:0/2374241207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198815c260 0x7f198815fe10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 -- 172.21.15.181:0/2374241207 >> 172.21.15.181:0/2374241207 conn(0x7f19880cebf0 msgr2=0x7f19880cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 -- 172.21.15.181:0/2374241207 shutdown_connections 2024-03-29T04:09:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.430+0000 7f198ea14640 1 -- 172.21.15.181:0/2374241207 wait complete. 2024-03-29T04:09:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.431+0000 7f198ea14640 1 Processor -- start 2024-03-29T04:09:17.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 -- start start 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19880b4300 0x7f19881e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 0x7f19880c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19880ce640 con 0x7f19880cc7c0 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f198815c640 con 0x7f19880b4300 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198ea14640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19880ce7c0 con 0x7f198815c260 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f19877fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f19877fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57760/0 (socket says 172.21.15.181:57760) 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f19877fe640 1 -- 172.21.15.181:0/1969550924 learned_addr learned my addr 172.21.15.181:0/1969550924 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f198cfb1640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 0x7f19880c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.432+0000 7f1987fff640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19880b4300 0x7f19881e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 -- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19880b4300 msgr2=0x7f19881e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19880b4300 0x7f19881e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 -- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 msgr2=0x7f198815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 -- 172.21.15.181:0/1969550924 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881e6940 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f19877fe640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198cfb1640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 0x7f19880c62e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f198000efc0 tx=0x7f198000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980018020 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1980014480 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881e6c20 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980014870 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.433+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19880c5ae0 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.434+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19880c4e70 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.434+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1980005d10 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.435+0000 7f1984ff9640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 0x7f197406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.435+0000 7f1987fff640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 0x7f197406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.435+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f198008c5a0 con 0x7f198815c260 2024-03-29T04:09:17.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.435+0000 7f1987fff640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 0x7f197406ea20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1978036000 tx=0x7f197803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:17.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.440+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f19880c4e70 con 0x7f198815c260 2024-03-29T04:09:17.594 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:17.614+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f19880702d0 con 0x7f198815c260 2024-03-29T04:09:19.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.322+0000 7f1984ff9640 1 -- 172.21.15.181:0/1969550924 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f1980022030 con 0x7f198815c260 2024-03-29T04:09:19.303 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data_ec' created 2024-03-29T04:09:19.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 msgr2=0x7f197406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:19.311 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 0x7f197406ea20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1978036000 tx=0x7f197803b040 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 msgr2=0x7f19880c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 0x7f19880c62e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f198000efc0 tx=0x7f198000c490 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 shutdown_connections 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f197406c560 0x7f197406ea20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f198815c260 0x7f19880c62e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880cc7c0 0x7f198815aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 --2- 172.21.15.181:0/1969550924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19880b4300 0x7f19881e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 >> 172.21.15.181:0/1969550924 conn(0x7f19880cebf0 msgr2=0x7f198815d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.332+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 shutdown_connections 2024-03-29T04:09:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.333+0000 7f198ea14640 1 -- 172.21.15.181:0/1969550924 wait complete. 2024-03-29T04:09:19.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-29T04:09:19.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.456+0000 7f6cb0900640 1 Processor -- start 2024-03-29T04:09:19.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.456+0000 7f6cb0900640 1 -- start start 2024-03-29T04:09:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6cac072f80 0x7f6cac073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6cac0b61e0 0x7f6cac072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac0cdad0 0x7f6cac0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f6cac0d0100 con 0x7f6cac0b61e0 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6cac0738a0 con 0x7f6cac072f80 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6cb0900640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac15eb90 con 0x7f6cac0cdad0 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6cac0b61e0 0x7f6cac072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6caa59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6cac072f80 0x7f6cac073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6caad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac0cdad0 0x7f6cac0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6cac0b61e0 0x7f6cac072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57556/0 (socket says 172.21.15.181:57556) 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 -- 172.21.15.181:0/3210219321 learned_addr learned my addr 172.21.15.181:0/3210219321 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 -- 172.21.15.181:0/3210219321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac0cdad0 msgr2=0x7f6cac0cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 --2- 172.21.15.181:0/3210219321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac0cdad0 0x7f6cac0cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 -- 172.21.15.181:0/3210219321 >> v2:172.21.15.187:3300/0 conn(0x7f6cac072f80 msgr2=0x7f6cac073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 --2- 172.21.15.181:0/3210219321 >> v2:172.21.15.187:3300/0 conn(0x7f6cac072f80 0x7f6cac073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.457+0000 7f6ca3fff640 1 -- 172.21.15.181:0/3210219321 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac0d03a0 con 0x7f6cac0b61e0 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6ca3fff640 1 --2- 172.21.15.181:0/3210219321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0b61e0 0x7f6cac072570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cac073d10 tx=0x7f6c9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50f1b62b164ac98 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6ca9d9b640 1 -- 172.21.15.181:0/3210219321 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9402e070 con 0x7f6cac0b61e0 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6ca9d9b640 1 -- 172.21.15.181:0/3210219321 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c94037070 con 0x7f6cac0b61e0 2024-03-29T04:09:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6ca9d9b640 1 -- 172.21.15.181:0/3210219321 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94031a00 con 0x7f6cac0b61e0 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6cb0900640 1 -- 172.21.15.181:0/3210219321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0b61e0 msgr2=0x7f6cac072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.458+0000 7f6cb0900640 1 --2- 172.21.15.181:0/3210219321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0b61e0 0x7f6cac072570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6cac073d10 tx=0x7f6c9402b880 comp rx=0 tx=0).stop 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 -- 172.21.15.181:0/3210219321 shutdown_connections 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 --2- 172.21.15.181:0/3210219321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac0cdad0 0x7f6cac0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 --2- 172.21.15.181:0/3210219321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0b61e0 0x7f6cac072570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 --2- 172.21.15.181:0/3210219321 >> v2:172.21.15.187:3300/0 conn(0x7f6cac072f80 0x7f6cac073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 -- 172.21.15.181:0/3210219321 >> 172.21.15.181:0/3210219321 conn(0x7f6cac0d10e0 msgr2=0x7f6cac0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 -- 172.21.15.181:0/3210219321 shutdown_connections 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.459+0000 7f6cb0900640 1 -- 172.21.15.181:0/3210219321 wait complete. 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.460+0000 7f6cb0900640 1 Processor -- start 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.460+0000 7f6cb0900640 1 -- start start 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac072f80 0x7f6cac14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cac0b61e0 0x7f6cac1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac072ab0 con 0x7f6cac0cdad0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cac0d0960 con 0x7f6cac0b61e0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cac0ce460 con 0x7f6cac072f80 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57558/0 (socket says 172.21.15.181:57558) 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 -- 172.21.15.181:0/1877334022 learned_addr learned my addr 172.21.15.181:0/1877334022 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caa59c640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac072f80 0x7f6cac14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6ca3fff640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cac0b61e0 0x7f6cac1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 -- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac072f80 msgr2=0x7f6cac14dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac072f80 0x7f6cac14dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 -- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cac0b61e0 msgr2=0x7f6cac1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cac0b61e0 0x7f6cac1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 -- 172.21.15.181:0/1877334022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cac1576e0 con 0x7f6cac0cdad0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6caad9d640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400efc0 tx=0x7f6ca400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4018020 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca4014480 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cac1579c0 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4014870 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.461+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cac157f80 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.462+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cac075900 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.463+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ca4005d10 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.463+0000 7f6ca2ffd640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 0x7f6c8c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.463+0000 7f6caa59c640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 0x7f6c8c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.463+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f6ca408c280 con 0x7f6cac0cdad0 2024-03-29T04:09:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.464+0000 7f6caa59c640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 0x7f6c8c06ea20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cac161ed0 tx=0x7f6c9c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.468+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6ca4022070 con 0x7f6cac0cdad0 2024-03-29T04:09:19.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:19.642+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6cac0d1ff0 con 0x7f6cac0cdad0 2024-03-29T04:09:20.158 INFO:tasks.ceph.mds.b.smithi187.stderr:2024-03-29T04:09:20.179+0000 7f8772d76640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:09:20.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.323+0000 7f6ca2ffd640 1 -- 172.21.15.181:0/1877334022 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f6ca4010040 con 0x7f6cac0cdad0 2024-03-29T04:09:20.303 INFO:teuthology.orchestra.run.smithi181.stderr:set pool 4 allow_ec_overwrites to true 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.328+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 msgr2=0x7f6c8c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.328+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 0x7f6c8c06ea20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cac161ed0 tx=0x7f6c9c009190 comp rx=0 tx=0).stop 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.328+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 msgr2=0x7f6cac14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.328+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400efc0 tx=0x7f6ca400c490 comp rx=0 tx=0).stop 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 shutdown_connections 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cac0cdad0 0x7f6cac14d0a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cac0b61e0 0x7f6cac1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6c8c06c560 0x7f6c8c06ea20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 --2- 172.21.15.181:0/1877334022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cac072f80 0x7f6cac14dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 >> 172.21.15.181:0/1877334022 conn(0x7f6cac0d10e0 msgr2=0x7f6cac06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:20.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 shutdown_connections 2024-03-29T04:09:20.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.329+0000 7f6cb0900640 1 -- 172.21.15.181:0/1877334022 wait complete. 2024-03-29T04:09:20.334 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-29T04:09:20.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.459+0000 7fbeb7ed3640 1 Processor -- start 2024-03-29T04:09:20.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 -- start start 2024-03-29T04:09:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbeb00b5dc0 0x7fbeb00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbeb00ba090 0x7fbeb00d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb00b66e0 con 0x7fbeb006c620 2024-03-29T04:09:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb00d2ff0 con 0x7fbeb00ba090 2024-03-29T04:09:20.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.460+0000 7fbeb7ed3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbeb00d32d0 con 0x7fbeb00b5dc0 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbeb00ba090 0x7fbeb00d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbeb00ba090 0x7fbeb00d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57566/0 (socket says 172.21.15.181:57566) 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 -- 172.21.15.181:0/158894336 learned_addr learned my addr 172.21.15.181:0/158894336 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb5c6f640 1 --2- 172.21.15.181:0/158894336 >> v2:172.21.15.187:3300/0 conn(0x7fbeb00b5dc0 0x7fbeb00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 -- 172.21.15.181:0/158894336 >> v2:172.21.15.187:3300/0 conn(0x7fbeb00b5dc0 msgr2=0x7fbeb00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 --2- 172.21.15.181:0/158894336 >> v2:172.21.15.187:3300/0 conn(0x7fbeb00b5dc0 0x7fbeb00b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 -- 172.21.15.181:0/158894336 >> v1:172.21.15.187:6789/0 conn(0x7fbeb006c620 legacy=0x7fbeb006ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.461+0000 7fbeb546e640 1 -- 172.21.15.181:0/158894336 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb00d35b0 con 0x7fbeb00ba090 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb546e640 1 --2- 172.21.15.181:0/158894336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00ba090 0x7fbeb00d2680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb006d770 tx=0x7fbeac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db28f965d17abac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb4c6d640 1 -- 172.21.15.181:0/158894336 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac034070 con 0x7fbeb00ba090 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb4c6d640 1 -- 172.21.15.181:0/158894336 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac030860 con 0x7fbeb00ba090 2024-03-29T04:09:20.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb4c6d640 1 -- 172.21.15.181:0/158894336 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac030c30 con 0x7fbeb00ba090 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/158894336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00ba090 msgr2=0x7fbeb00d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/158894336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00ba090 0x7fbeb00d2680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbeb006d770 tx=0x7fbeac02b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/158894336 shutdown_connections 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/158894336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00ba090 0x7fbeb00d2680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/158894336 >> v2:172.21.15.187:3300/0 conn(0x7fbeb00b5dc0 0x7fbeb00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/158894336 >> 172.21.15.181:0/158894336 conn(0x7fbeb00678f0 msgr2=0x7fbeb00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/158894336 shutdown_connections 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.462+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/158894336 wait complete. 2024-03-29T04:09:20.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.463+0000 7fbeb7ed3640 1 Processor -- start 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 -- start start 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeb00b5dc0 0x7fbeb0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbeb00ba090 0x7fbeb00ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb00d4090 con 0x7fbeb006c620 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb00d3b70 con 0x7fbeb00b5dc0 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb7ed3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb00b83e0 con 0x7fbeb00ba090 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb5c6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb5c6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57580/0 (socket says 172.21.15.181:57580) 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb5c6f640 1 -- 172.21.15.181:0/2989710954 learned_addr learned my addr 172.21.15.181:0/2989710954 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb546e640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeb00b5dc0 0x7fbeb0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.464+0000 7fbeb6470640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbeb00ba090 0x7fbeb00ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 -- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbeb00ba090 msgr2=0x7fbeb00ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbeb00ba090 0x7fbeb00ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 -- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeb00b5dc0 msgr2=0x7fbeb0157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeb00b5dc0 0x7fbeb0157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 -- 172.21.15.181:0/2989710954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01f7da0 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb5c6f640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbea000e9b0 tx=0x7fbea000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0017070 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea000c9a0 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb01f8080 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea000cd90 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.465+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01f8640 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.466+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb0075890 con 0x7fbeb006c620 2024-03-29T04:09:20.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.466+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbea001a070 con 0x7fbeb006c620 2024-03-29T04:09:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.467+0000 7fbea67fc640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 0x7fbe9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.467+0000 7fbeb546e640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 0x7fbe9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.467+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fbea008bcb0 con 0x7fbeb006c620 2024-03-29T04:09:20.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.467+0000 7fbeb546e640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 0x7fbe9406e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbeac036000 tx=0x7fbeac03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:20.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.471+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbea00599f0 con 0x7fbeb006c620 2024-03-29T04:09:20.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:20.684+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fbeb00cf040 con 0x7fbeb006c620 2024-03-29T04:09:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.329+0000 7fbea67fc640 1 -- 172.21.15.181:0/2989710954 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fbea005dcf0 con 0x7fbeb006c620 2024-03-29T04:09:21.309 INFO:teuthology.orchestra.run.smithi181.stderr:added data pool 4 to fsmap 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 msgr2=0x7fbe9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 0x7fbe9406e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbeac036000 tx=0x7fbeac03b040 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 msgr2=0x7fbeb015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbea000e9b0 tx=0x7fbea000ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 shutdown_connections 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe9406c530 0x7fbe9406e9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbeb00ba090 0x7fbeb00ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeb00b5dc0 0x7fbeb0157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 --2- 172.21.15.181:0/2989710954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb006c620 0x7fbeb015fda0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 >> 172.21.15.181:0/2989710954 conn(0x7fbeb00678f0 msgr2=0x7fbeb00b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 shutdown_connections 2024-03-29T04:09:21.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.332+0000 7fbeb7ed3640 1 -- 172.21.15.181:0/2989710954 wait complete. 2024-03-29T04:09:21.325 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.452+0000 7f03404b7640 1 Processor -- start 2024-03-29T04:09:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.452+0000 7f03404b7640 1 -- start start 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f03404b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f03404b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f03404b7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03380724e0 con 0x7f0338071fb0 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f03404b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f03380730f0 con 0x7f03380cd3b0 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f03404b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03380727c0 con 0x7f033815a960 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033da52640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:36500/0 (socket says 172.21.15.181:36500) 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033da52640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:47184/0 (socket says 172.21.15.181:47184) 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 -- 172.21.15.181:0/1718158865 learned_addr learned my addr 172.21.15.181:0/1718158865 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033ea54640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0338071fb0 0x7f033815e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:52844/0 (socket says 172.21.15.181:52844) 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 -- 172.21.15.181:0/1718158865 >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 msgr2=0x7f03380cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 --2- 172.21.15.181:0/1718158865 >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 -- 172.21.15.181:0/1718158865 >> v1:172.21.15.187:6789/0 conn(0x7f0338071fb0 legacy=0x7f033815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033e253640 1 -- 172.21.15.181:0/1718158865 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0338072aa0 con 0x7f033815a960 2024-03-29T04:09:21.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.453+0000 7f033da52640 1 --2- 172.21.15.181:0/1718158865 >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.454+0000 7f033e253640 1 --2- 172.21.15.181:0/1718158865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f033000d150 tx=0x7f033002e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ddba911c90dd84 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.454+0000 7f033d251640 1 -- 172.21.15.181:0/1718158865 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033000b0d0 con 0x7f033815a960 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.454+0000 7f033d251640 1 -- 172.21.15.181:0/1718158865 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033000b270 con 0x7f033815a960 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.454+0000 7f033d251640 1 -- 172.21.15.181:0/1718158865 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330009600 con 0x7f033815a960 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 -- 172.21.15.181:0/1718158865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 msgr2=0x7f0338071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 --2- 172.21.15.181:0/1718158865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f033000d150 tx=0x7f033002e160 comp rx=0 tx=0).stop 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 -- 172.21.15.181:0/1718158865 shutdown_connections 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 --2- 172.21.15.181:0/1718158865 >> v2:172.21.15.187:3300/0 conn(0x7f03380cd3b0 0x7f03380cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 --2- 172.21.15.181:0/1718158865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033815a960 0x7f0338071a70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 -- 172.21.15.181:0/1718158865 >> 172.21.15.181:0/1718158865 conn(0x7f03380b1c90 msgr2=0x7f03380b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:21.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 -- 172.21.15.181:0/1718158865 shutdown_connections 2024-03-29T04:09:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.455+0000 7f03404b7640 1 -- 172.21.15.181:0/1718158865 wait complete. 2024-03-29T04:09:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.456+0000 7f03404b7640 1 Processor -- start 2024-03-29T04:09:21.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.456+0000 7f03404b7640 1 -- start start 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 0x7f033814cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03380cd3b0 0x7f03380c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03380cede0 con 0x7f033815a960 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f033815ad40 con 0x7f03380cd3b0 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f03404b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03380cef60 con 0x7f0338071fb0 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033da52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03380cd3b0 0x7f03380c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57588/0 (socket says 172.21.15.181:57588) 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 -- 172.21.15.181:0/508836942 learned_addr learned my addr 172.21.15.181:0/508836942 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:21.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033e253640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 0x7f033814cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 -- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 msgr2=0x7f033814cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 0x7f033814cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 -- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03380cd3b0 msgr2=0x7f03380c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03380cd3b0 0x7f03380c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 -- 172.21.15.181:0/508836942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03380c77c0 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033e253640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 0x7f033814cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f033ea54640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f032800c430 tx=0x7f032800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.457+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280074e0 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.458+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328007680 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.458+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03380c7a30 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.458+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280049e0 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.458+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03380c7ff0 con 0x7f033815a960 2024-03-29T04:09:21.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.459+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0338075a10 con 0x7f033815a960 2024-03-29T04:09:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.459+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f032801a020 con 0x7f033815a960 2024-03-29T04:09:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.459+0000 7f032effd640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 0x7f031c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.459+0000 7f033e253640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 0x7f031c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.459+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f032808b090 con 0x7f033815a960 2024-03-29T04:09:21.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.460+0000 7f033e253640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 0x7f031c06e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0330037000 tx=0x7f0330032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.464+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0328058d20 con 0x7f033815a960 2024-03-29T04:09:21.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.679+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f033806e2e0 con 0x7f033815a960 2024-03-29T04:09:21.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.679+0000 7f032effd640 1 -- 172.21.15.181:0/508836942 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f032805f070 con 0x7f033815a960 2024-03-29T04:09:21.659 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:21.659 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:20.686536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:21.660 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 7 2024-03-29T04:09:21.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 msgr2=0x7f031c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 0x7f031c06e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0330037000 tx=0x7f0330032000 comp rx=0 tx=0).stop 2024-03-29T04:09:21.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 msgr2=0x7f03380c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f032800c430 tx=0x7f032800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 shutdown_connections 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033815a960 0x7f03380c9bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03380cd3b0 0x7f03380c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f031c06c530 0x7f031c06e9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 --2- 172.21.15.181:0/508836942 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0338071fb0 0x7f033814cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 >> 172.21.15.181:0/508836942 conn(0x7f03380b1c90 msgr2=0x7f0338159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 shutdown_connections 2024-03-29T04:09:21.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.681+0000 7f03404b7640 1 -- 172.21.15.181:0/508836942 wait complete. 2024-03-29T04:09:21.672 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:21.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.793+0000 7f75f5481640 1 Processor -- start 2024-03-29T04:09:21.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.793+0000 7f75f5481640 1 -- start start 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75f5481640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75f5481640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 0x7f75f01624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75f5481640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f0141910 con 0x7f75f00b5240 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75f5481640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f75f0071e60 con 0x7f75f0162030 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75f5481640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75f0162af0 con 0x7f75f0141530 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75ee7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 0x7f75f01624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75eeffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:36520/0 (socket says 172.21.15.181:36520) 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75ee7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 0x7f75f01624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:47192/0 (socket says 172.21.15.181:47192) 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.794+0000 7f75eeffd640 1 -- 172.21.15.181:0/1511424771 learned_addr learned my addr 172.21.15.181:0/1511424771 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:21.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75eeffd640 1 -- 172.21.15.181:0/1511424771 >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 msgr2=0x7f75f01624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75eeffd640 1 --2- 172.21.15.181:0/1511424771 >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 0x7f75f01624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75eeffd640 1 -- 172.21.15.181:0/1511424771 >> v1:172.21.15.187:6789/0 conn(0x7f75f00b5240 legacy=0x7f75f0141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75eeffd640 1 -- 172.21.15.181:0/1511424771 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f0162dd0 con 0x7f75f0141530 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75eeffd640 1 --2- 172.21.15.181:0/1511424771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f75f0141ae0 tx=0x7f75e002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd2ebcd0da5c5ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75edffb640 1 -- 172.21.15.181:0/1511424771 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e002ca50 con 0x7f75f0141530 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75edffb640 1 -- 172.21.15.181:0/1511424771 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e002cbf0 con 0x7f75f0141530 2024-03-29T04:09:21.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.795+0000 7f75edffb640 1 -- 172.21.15.181:0/1511424771 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e00343f0 con 0x7f75f0141530 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 -- 172.21.15.181:0/1511424771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 msgr2=0x7f75f0071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 --2- 172.21.15.181:0/1511424771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f75f0141ae0 tx=0x7f75e002beb0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 -- 172.21.15.181:0/1511424771 shutdown_connections 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 --2- 172.21.15.181:0/1511424771 >> v2:172.21.15.187:3300/0 conn(0x7f75f0162030 0x7f75f01624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 --2- 172.21.15.181:0/1511424771 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0071920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 -- 172.21.15.181:0/1511424771 >> 172.21.15.181:0/1511424771 conn(0x7f75f006f8b0 msgr2=0x7f75f006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 -- 172.21.15.181:0/1511424771 shutdown_connections 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.796+0000 7f75f5481640 1 -- 172.21.15.181:0/1511424771 wait complete. 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.797+0000 7f75f5481640 1 Processor -- start 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.797+0000 7f75f5481640 1 -- start start 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 0x7f75f013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f75f00b1950 con 0x7f75f0162030 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0071400 con 0x7f75f00b5240 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75f5481640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75f0142320 con 0x7f75f0141530 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75ef7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57596/0 (socket says 172.21.15.181:57596) 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75ef7fe640 1 -- 172.21.15.181:0/3336757246 learned_addr learned my addr 172.21.15.181:0/3336757246 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75eeffd640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 0x7f75f013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.798+0000 7f75ee7fc640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 -- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 msgr2=0x7f75f0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 -- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 msgr2=0x7f75f013d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 0x7f75f013d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 -- 172.21.15.181:0/3336757246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f01f5300 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75eeffd640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 0x7f75f013d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75ef7fe640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75e8001eb0 tx=0x7f75e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017070 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e80052d0 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8003810 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f0158f20 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.799+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01594e0 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.800+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f0075a10 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.801+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f75e801a070 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.802+0000 7f75d77fe640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 0x7f75dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.802+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f75e808aca0 con 0x7f75f0162030 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.802+0000 7f75eeffd640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 0x7f75dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:21.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.802+0000 7f75eeffd640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 0x7f75dc06e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75e0035000 tx=0x7f75e0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:21.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.806+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f75e80589c0 con 0x7f75f0162030 2024-03-29T04:09:21.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.983+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75f00d2f00 con 0x7f75f0162030 2024-03-29T04:09:21.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.984+0000 7f75d77fe640 1 -- 172.21.15.181:0/3336757246 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18539 (secure 0 0 0) 0x7f75e805ccc0 con 0x7f75f0162030 2024-03-29T04:09:21.964 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:21.964 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":24,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:20.686294+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 msgr2=0x7f75dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 0x7f75dc06e9f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75e0035000 tx=0x7f75e0030000 comp rx=0 tx=0).stop 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 msgr2=0x7f75f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75e8001eb0 tx=0x7f75e800c890 comp rx=0 tx=0).stop 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 shutdown_connections 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f0162030 0x7f75f006ccf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f75dc06c530 0x7f75dc06e9f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75f0141530 0x7f75f0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 --2- 172.21.15.181:0/3336757246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f00b5240 0x7f75f013d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:21.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 >> 172.21.15.181:0/3336757246 conn(0x7f75f006f8b0 msgr2=0x7f75f0161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:21.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 shutdown_connections 2024-03-29T04:09:21.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:21.986+0000 7f75f5481640 1 -- 172.21.15.181:0/3336757246 wait complete. 2024-03-29T04:09:21.977 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.097+0000 7fca4c5e8640 1 Processor -- start 2024-03-29T04:09:22.077 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.097+0000 7fca4c5e8640 1 -- start start 2024-03-29T04:09:22.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4c5e8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca440cd3b0 0x7fca440cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4c5e8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca440724e0 con 0x7fca4415a960 2024-03-29T04:09:22.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4c5e8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca440727c0 con 0x7fca440cb3b0 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4c5e8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fca440730f0 con 0x7fca440cd3b0 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca49b83640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca440cd3b0 0x7fca440cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4a384640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fca4415a960 0x7fca44071a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32782/0 (socket says 172.21.15.181:32782) 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.098+0000 7fca4a384640 1 -- 172.21.15.181:0/2421162650 learned_addr learned my addr 172.21.15.181:0/2421162650 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.099+0000 7fca49b83640 1 -- 172.21.15.181:0/2421162650 >> v1:172.21.15.187:6789/0 conn(0x7fca440cb3b0 legacy=0x7fca440cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.099+0000 7fca49b83640 1 -- 172.21.15.181:0/2421162650 >> v1:172.21.15.181:6789/0 conn(0x7fca4415a960 legacy=0x7fca44071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.099+0000 7fca49382640 1 -- 172.21.15.181:0/2421162650 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002935923 0 0) 0x7fca440727c0 con 0x7fca440cb3b0 2024-03-29T04:09:22.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.099+0000 7fca49b83640 1 -- 172.21.15.181:0/2421162650 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca44072aa0 con 0x7fca440cd3b0 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.099+0000 7fca49b83640 1 --2- 172.21.15.181:0/2421162650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca440cd3b0 0x7fca440cae70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca400097a0 tx=0x7fca4002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63003d58bd77b2e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca49382640 1 -- 172.21.15.181:0/2421162650 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40037070 con 0x7fca440cd3b0 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca49382640 1 -- 172.21.15.181:0/2421162650 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca40030710 con 0x7fca440cd3b0 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca49382640 1 -- 172.21.15.181:0/2421162650 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40030ae0 con 0x7fca440cd3b0 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 -- 172.21.15.181:0/2421162650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca440cd3b0 msgr2=0x7fca440cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/2421162650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca440cd3b0 0x7fca440cae70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca400097a0 tx=0x7fca4002b790 comp rx=0 tx=0).stop 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 -- 172.21.15.181:0/2421162650 shutdown_connections 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/2421162650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca440cd3b0 0x7fca440cae70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 -- 172.21.15.181:0/2421162650 >> 172.21.15.181:0/2421162650 conn(0x7fca440b1c90 msgr2=0x7fca440b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 -- 172.21.15.181:0/2421162650 shutdown_connections 2024-03-29T04:09:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.100+0000 7fca4c5e8640 1 -- 172.21.15.181:0/2421162650 wait complete. 2024-03-29T04:09:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.101+0000 7fca4c5e8640 1 Processor -- start 2024-03-29T04:09:22.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.101+0000 7fca4c5e8640 1 -- start start 2024-03-29T04:09:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca440cb3b0 0x7fca44154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca440cd3b0 0x7fca44159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca44072180 con 0x7fca4415a960 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca4415ad40 con 0x7fca440cb3b0 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4c5e8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fca440cca40 con 0x7fca440cd3b0 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57604/0 (socket says 172.21.15.181:57604) 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 -- 172.21.15.181:0/3567734756 learned_addr learned my addr 172.21.15.181:0/3567734756 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4a384640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca440cb3b0 0x7fca44154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca49b83640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca440cd3b0 0x7fca44159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 -- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca440cd3b0 msgr2=0x7fca44159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca440cd3b0 0x7fca44159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 -- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca440cb3b0 msgr2=0x7fca44154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca440cb3b0 0x7fca44154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.102+0000 7fca4ab85640 1 -- 172.21.15.181:0/3567734756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca44151ed0 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca4ab85640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00c790 tx=0x7fca3c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003da0 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c005070 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca44150e10 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c005460 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.103+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca441513d0 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.104+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca44157360 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.104+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fca3c005600 con 0x7fca4415a960 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.104+0000 7fca36ffd640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 0x7fca2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.105+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fca3c08b760 con 0x7fca4415a960 2024-03-29T04:09:22.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.105+0000 7fca4a384640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 0x7fca2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.105+0000 7fca4a384640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 0x7fca2806e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca38000f60 tx=0x7fca38007e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.109+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fca3c059480 con 0x7fca4415a960 2024-03-29T04:09:22.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.285+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca44075a10 con 0x7fca4415a960 2024-03-29T04:09:22.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.286+0000 7fca36ffd640 1 -- 172.21.15.181:0/3567734756 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18539 (secure 0 0 0) 0x7fca3c05d780 con 0x7fca4415a960 2024-03-29T04:09:22.266 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:22.266 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":24,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:20.686294+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 msgr2=0x7fca2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 0x7fca2806e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca38000f60 tx=0x7fca38007e20 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 msgr2=0x7fca441558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00c790 tx=0x7fca3c00cc50 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 shutdown_connections 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca4415a960 0x7fca441558c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fca2806c530 0x7fca2806e9f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fca440cd3b0 0x7fca44159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 --2- 172.21.15.181:0/3567734756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca440cb3b0 0x7fca44154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 >> 172.21.15.181:0/3567734756 conn(0x7fca440b1c90 msgr2=0x7fca440b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 shutdown_connections 2024-03-29T04:09:22.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.288+0000 7fca4c5e8640 1 -- 172.21.15.181:0/3567734756 wait complete. 2024-03-29T04:09:22.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:22.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.417+0000 7f88b4eb2640 1 Processor -- start 2024-03-29T04:09:22.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.417+0000 7f88b4eb2640 1 -- start start 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88b4eb2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 0x7f88b015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88b4eb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 0x7f88b00ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88b4eb2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b00d0730 con 0x7f88b0158ac0 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88b4eb2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f88b015fab0 con 0x7f88b00b43e0 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88b4eb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88b015f7d0 con 0x7f88b015f000 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88ae59c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88b0158ac0 0x7f88b00b3cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32794/0 (socket says 172.21.15.181:32794) 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88ae59c640 1 -- 172.21.15.181:0/57809176 learned_addr learned my addr 172.21.15.181:0/57809176 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88add9b640 1 --2- 172.21.15.181:0/57809176 >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 0x7f88b015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88aed9d640 1 --2- 172.21.15.181:0/57809176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 0x7f88b00ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88ad59a640 1 -- 172.21.15.181:0/57809176 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007288031 0 0) 0x7f88b00d0730 con 0x7f88b0158ac0 2024-03-29T04:09:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88ad59a640 1 -- 172.21.15.181:0/57809176 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8898003520 con 0x7f88b0158ac0 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88aed9d640 1 -- 172.21.15.181:0/57809176 >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 msgr2=0x7f88b015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88aed9d640 1 --2- 172.21.15.181:0/57809176 >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 0x7f88b015eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88aed9d640 1 -- 172.21.15.181:0/57809176 >> v1:172.21.15.181:6789/0 conn(0x7f88b0158ac0 legacy=0x7f88b00b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.418+0000 7f88aed9d640 1 -- 172.21.15.181:0/57809176 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b006c6e0 con 0x7f88b015f000 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.419+0000 7f88add9b640 1 --2- 172.21.15.181:0/57809176 >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 0x7f88b015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.419+0000 7f88aed9d640 1 --2- 172.21.15.181:0/57809176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 0x7f88b00ce8c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d90 tx=0x7f88a802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2531f2474f6b06f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.419+0000 7f88ad59a640 1 -- 172.21.15.181:0/57809176 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8039070 con 0x7f88b015f000 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.419+0000 7f88ad59a640 1 -- 172.21.15.181:0/57809176 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a802cbc0 con 0x7f88b015f000 2024-03-29T04:09:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.419+0000 7f88ad59a640 1 -- 172.21.15.181:0/57809176 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8034440 con 0x7f88b015f000 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 -- 172.21.15.181:0/57809176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 msgr2=0x7f88b00ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/57809176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 0x7f88b00ce8c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d90 tx=0x7f88a802be10 comp rx=0 tx=0).stop 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 -- 172.21.15.181:0/57809176 shutdown_connections 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/57809176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b015f000 0x7f88b00ce8c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/57809176 >> v2:172.21.15.187:3300/0 conn(0x7f88b00b43e0 0x7f88b015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 -- 172.21.15.181:0/57809176 >> 172.21.15.181:0/57809176 conn(0x7f88b006f820 msgr2=0x7f88b006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 -- 172.21.15.181:0/57809176 shutdown_connections 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.420+0000 7f88b4eb2640 1 -- 172.21.15.181:0/57809176 wait complete. 2024-03-29T04:09:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.421+0000 7f88b4eb2640 1 Processor -- start 2024-03-29T04:09:22.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.421+0000 7f88b4eb2640 1 -- start start 2024-03-29T04:09:22.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b00b43e0 0x7f88b0155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88b015f000 0x7f88b015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88b015f490 con 0x7f88b0158ac0 2024-03-29T04:09:22.404 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88b006cca0 con 0x7f88b015f000 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88b4eb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88b0158ea0 con 0x7f88b00b43e0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57620/0 (socket says 172.21.15.181:57620) 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 -- 172.21.15.181:0/3223923056 learned_addr learned my addr 172.21.15.181:0/3223923056 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88ae59c640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b00b43e0 0x7f88b0155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88aed9d640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88b015f000 0x7f88b015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 -- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b00b43e0 msgr2=0x7f88b0155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b00b43e0 0x7f88b0155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 -- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88b015f000 msgr2=0x7f88b015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88b015f000 0x7f88b015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.422+0000 7f88add9b640 1 -- 172.21.15.181:0/3223923056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b00afda0 con 0x7f88b0158ac0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f88add9b640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ebb0 tx=0x7f88a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4017070 con 0x7f88b0158ac0 2024-03-29T04:09:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a400cc30 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b00b0080 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4013470 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.423+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b00b0640 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.424+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b0075a10 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.424+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f88a401a070 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.425+0000 7f889effd640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 0x7f889006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.425+0000 7f88ae59c640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 0x7f889006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.425+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f88a408bc30 con 0x7f88b0158ac0 2024-03-29T04:09:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.425+0000 7f88ae59c640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 0x7f889006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005880 tx=0x7f88a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.429+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f88a4059950 con 0x7f88b0158ac0 2024-03-29T04:09:22.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.642+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88b0155fb0 con 0x7f88b0158ac0 2024-03-29T04:09:22.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.642+0000 7f889effd640 1 -- 172.21.15.181:0/3223923056 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f88a405dc50 con 0x7f88b0158ac0 2024-03-29T04:09:22.626 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:22.626 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:20.686536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:22.627 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 7 2024-03-29T04:09:22.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 msgr2=0x7f889006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 0x7f889006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005880 tx=0x7f88a0009040 comp rx=0 tx=0).stop 2024-03-29T04:09:22.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 msgr2=0x7f88b00b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ebb0 tx=0x7f88a400c520 comp rx=0 tx=0).stop 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 shutdown_connections 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88b015f000 0x7f88b015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.644+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88b0158ac0 0x7f88b00b1120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.645+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f889006c530 0x7f889006e9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.645+0000 7f88b4eb2640 1 --2- 172.21.15.181:0/3223923056 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88b00b43e0 0x7f88b0155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.645+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 >> 172.21.15.181:0/3223923056 conn(0x7f88b006f820 msgr2=0x7f88b01612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.645+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 shutdown_connections 2024-03-29T04:09:22.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.645+0000 7f88b4eb2640 1 -- 172.21.15.181:0/3223923056 wait complete. 2024-03-29T04:09:22.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:22.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.757+0000 7fd6d4e55640 1 Processor -- start 2024-03-29T04:09:22.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.757+0000 7fd6d4e55640 1 -- start start 2024-03-29T04:09:22.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6d4e55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 0x7fd6d015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6d4e55640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd6d00b4300 0x7fd6d0144580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6d4e55640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d015fbe0 con 0x7fd6d0141da0 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6d4e55640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d00d1cd0 con 0x7fd6d00b3f20 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6d4e55640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d01625f0 con 0x7fd6d00b4300 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ce59c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 0x7fd6d015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6cdd9b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd6d0141da0 0x7fd6d00d2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32798/0 (socket says 172.21.15.181:32798) 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6cdd9b640 1 -- 172.21.15.181:0/1618391266 learned_addr learned my addr 172.21.15.181:0/1618391266 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ced9d640 1 --2- 172.21.15.181:0/1618391266 >> v2:172.21.15.187:3300/0 conn(0x7fd6d00b4300 0x7fd6d0144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6cd59a640 1 -- 172.21.15.181:0/1618391266 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616767966 0 0) 0x7fd6c4001110 con 0x7fd6d0141da0 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6cd59a640 1 -- 172.21.15.181:0/1618391266 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b8003520 con 0x7fd6d0141da0 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ced9d640 1 -- 172.21.15.181:0/1618391266 >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 msgr2=0x7fd6d015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ced9d640 1 --2- 172.21.15.181:0/1618391266 >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 0x7fd6d015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ced9d640 1 -- 172.21.15.181:0/1618391266 >> v1:172.21.15.181:6789/0 conn(0x7fd6d0141da0 legacy=0x7fd6d00d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ced9d640 1 -- 172.21.15.181:0/1618391266 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01628d0 con 0x7fd6d00b4300 2024-03-29T04:09:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.758+0000 7fd6ce59c640 1 --2- 172.21.15.181:0/1618391266 >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 0x7fd6d015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.759+0000 7fd6ced9d640 1 --2- 172.21.15.181:0/1618391266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d0144580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00b8360 tx=0x7fd6c802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f10160a35ef26a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.759+0000 7fd6cd59a640 1 -- 172.21.15.181:0/1618391266 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8039070 con 0x7fd6d00b4300 2024-03-29T04:09:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.759+0000 7fd6cd59a640 1 -- 172.21.15.181:0/1618391266 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c802cc80 con 0x7fd6d00b4300 2024-03-29T04:09:22.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.759+0000 7fd6cd59a640 1 -- 172.21.15.181:0/1618391266 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8034660 con 0x7fd6d00b4300 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 -- 172.21.15.181:0/1618391266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 msgr2=0x7fd6d0144580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/1618391266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d0144580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00b8360 tx=0x7fd6c802bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 -- 172.21.15.181:0/1618391266 shutdown_connections 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/1618391266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d0144580 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/1618391266 >> v2:172.21.15.181:3300/0 conn(0x7fd6d00b3f20 0x7fd6d015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 -- 172.21.15.181:0/1618391266 >> 172.21.15.181:0/1618391266 conn(0x7fd6d00672c0 msgr2=0x7fd6d00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 -- 172.21.15.181:0/1618391266 shutdown_connections 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.760+0000 7fd6d4e55640 1 -- 172.21.15.181:0/1618391266 wait complete. 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.761+0000 7fd6d4e55640 1 Processor -- start 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.761+0000 7fd6d4e55640 1 -- start start 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6d0141da0 0x7fd6d00c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00d0060 con 0x7fd6d00b3f20 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0162e90 con 0x7fd6d00b4300 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d0162210 con 0x7fd6d0141da0 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57642/0 (socket says 172.21.15.181:57642) 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 -- 172.21.15.181:0/3967034836 learned_addr learned my addr 172.21.15.181:0/3967034836 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6cdd9b640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ced9d640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6d0141da0 0x7fd6d00c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 -- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6d0141da0 msgr2=0x7fd6d00c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6d0141da0 0x7fd6d00c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 -- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 msgr2=0x7fd6d00ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d00ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 -- 172.21.15.181:0/3967034836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d015aa60 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6ce59c640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000ebe0 tx=0x7fd6c000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0017070 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c000cc30 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d00c9830 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0013470 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.762+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d00c9df0 con 0x7fd6d00b3f20 2024-03-29T04:09:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.763+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0075a10 con 0x7fd6d00b3f20 2024-03-29T04:09:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.764+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd6c001a070 con 0x7fd6d00b3f20 2024-03-29T04:09:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.764+0000 7fd6beffd640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 0x7fd6b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.764+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6c008b850 con 0x7fd6d00b3f20 2024-03-29T04:09:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.764+0000 7fd6cdd9b640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 0x7fd6b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.765+0000 7fd6cdd9b640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 0x7fd6b006e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6d006db90 tx=0x7fd6c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:22.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.769+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd6c0059570 con 0x7fd6d00b3f20 2024-03-29T04:09:22.925 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.945+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d0143a60 con 0x7fd6d00b3f20 2024-03-29T04:09:22.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.946+0000 7fd6beffd640 1 -- 172.21.15.181:0/3967034836 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fd6c005d870 con 0x7fd6d00b3f20 2024-03-29T04:09:22.926 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:22.926 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 msgr2=0x7fd6b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 0x7fd6b006e9f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6d006db90 tx=0x7fd6c4009040 comp rx=0 tx=0).stop 2024-03-29T04:09:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 msgr2=0x7fd6d00b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:22.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000ebe0 tx=0x7fd6c000c520 comp rx=0 tx=0).stop 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 shutdown_connections 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd6b006c530 0x7fd6b006e9f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6d0141da0 0x7fd6d00c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6d00b4300 0x7fd6d00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.948+0000 7fd6d4e55640 1 --2- 172.21.15.181:0/3967034836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6d00b3f20 0x7fd6d00b2e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.949+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 >> 172.21.15.181:0/3967034836 conn(0x7fd6d00672c0 msgr2=0x7fd6d00cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.949+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 shutdown_connections 2024-03-29T04:09:22.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:22.949+0000 7fd6d4e55640 1 -- 172.21.15.181:0/3967034836 wait complete. 2024-03-29T04:09:23.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:09:15.639711+0000', 'modified': '2024-03-29T04:09:20.686536+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/3068825067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 3068825067}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3068825067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-03-29T04:09:23.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:09:23.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T04:09:23.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.466+0000 7f1f0067e640 1 Processor -- start 2024-03-29T04:09:23.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.466+0000 7f1f0067e640 1 -- start start 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1f0067e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ef806cca0 0x7f1ef80c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1f0067e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef80cb510 con 0x7f1ef80afd70 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1f0067e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ef815cc10 con 0x7f1ef80c74a0 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1f0067e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef8156d30 con 0x7f1ef806cca0 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ef806cca0 0x7f1ef80c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1ef806cca0 0x7f1ef80c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:47268/0 (socket says 172.21.15.181:47268) 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efec1b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1ef80afd70 0x7f1ef80c5d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32810/0 (socket says 172.21.15.181:32810) 2024-03-29T04:09:23.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 -- 172.21.15.181:0/503801692 learned_addr learned my addr 172.21.15.181:0/503801692 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efe41a640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1ef80c74a0 0x7f1ef806c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:52864/0 (socket says 172.21.15.181:52864) 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901942784 0 0) 0x7f1ef80cb510 con 0x7f1ef80afd70 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ee4003520 con 0x7f1ef80afd70 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 -- 172.21.15.181:0/503801692 >> v1:172.21.15.187:6789/0 conn(0x7f1ef80c74a0 legacy=0x7f1ef806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 -- 172.21.15.181:0/503801692 >> v1:172.21.15.181:6789/0 conn(0x7f1ef80afd70 legacy=0x7f1ef80c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350398133 0 0) 0x7f1ef815cc10 con 0x7f1ef80c74a0 2024-03-29T04:09:23.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.467+0000 7f1efdc19640 1 -- 172.21.15.181:0/503801692 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef815cef0 con 0x7f1ef806cca0 2024-03-29T04:09:23.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.468+0000 7f1efdc19640 1 --2- 172.21.15.181:0/503801692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef806cca0 0x7f1ef80c57c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009870 tx=0x7f1ee802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec9fe57038cdb0a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.468+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8034070 con 0x7f1ef806cca0 2024-03-29T04:09:23.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.468+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee8030860 con 0x7f1ef806cca0 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.468+0000 7f1efd418640 1 -- 172.21.15.181:0/503801692 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8030c30 con 0x7f1ef806cca0 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 -- 172.21.15.181:0/503801692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef806cca0 msgr2=0x7f1ef80c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 --2- 172.21.15.181:0/503801692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef806cca0 0x7f1ef80c57c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009870 tx=0x7f1ee802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 -- 172.21.15.181:0/503801692 shutdown_connections 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 --2- 172.21.15.181:0/503801692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef806cca0 0x7f1ef80c57c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 -- 172.21.15.181:0/503801692 >> 172.21.15.181:0/503801692 conn(0x7f1ef8071d00 msgr2=0x7f1ef80720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 -- 172.21.15.181:0/503801692 shutdown_connections 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.469+0000 7f1f0067e640 1 -- 172.21.15.181:0/503801692 wait complete. 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.470+0000 7f1f0067e640 1 Processor -- start 2024-03-29T04:09:23.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.470+0000 7f1f0067e640 1 -- start start 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 0x7f1ef80b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef80afd70 0x7f1ef81f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef80cc420 con 0x7f1ef80c74a0 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef815c900 con 0x7f1ef80afd70 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef815d6f0 con 0x7f1ef806cca0 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efe41a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 0x7f1ef80b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57654/0 (socket says 172.21.15.181:57654) 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 -- 172.21.15.181:0/331269790 learned_addr learned my addr 172.21.15.181:0/331269790 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efdc19640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef80afd70 0x7f1ef81f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 -- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 msgr2=0x7f1ef80b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 0x7f1ef80b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 -- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef80afd70 msgr2=0x7f1ef81f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef80afd70 0x7f1ef81f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 -- 172.21.15.181:0/331269790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef8148640 con 0x7f1ef80c74a0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efe41a640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 0x7f1ef80b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1efec1b640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c430 tx=0x7f1ef000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1eeeffd640 1 -- 172.21.15.181:0/331269790 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00074e0 con 0x7f1ef80c74a0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1eeeffd640 1 -- 172.21.15.181:0/331269790 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0007680 con 0x7f1ef80c74a0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef8148920 con 0x7f1ef80c74a0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1eeeffd640 1 -- 172.21.15.181:0/331269790 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0004a20 con 0x7f1ef80c74a0 2024-03-29T04:09:23.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.471+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef8148ee0 con 0x7f1ef80c74a0 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.473+0000 7f1eeeffd640 1 -- 172.21.15.181:0/331269790 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1ef001a020 con 0x7f1ef80c74a0 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.473+0000 7f1eeeffd640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 0x7f1edc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.473+0000 7f1eeeffd640 1 -- 172.21.15.181:0/331269790 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ef008b0e0 con 0x7f1ef80c74a0 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.473+0000 7f1efe41a640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 0x7f1edc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.474+0000 7f1efe41a640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 0x7f1edc06e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40012e0 tx=0x7f1ef4001700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.474+0000 7f1f03260740 1 Processor -- start 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 -- start start 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556d4fa86020 con 0x556d4fb1cd40 2024-03-29T04:09:23.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d4fa2e5f0 con 0x556d4fb49c40 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556d4fa81fd0 con 0x556d4fb49860 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57668/0 (socket says 172.21.15.181:57668) 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1efec1b640 1 -- 172.21.15.181:0/2500032334 learned_addr learned my addr 172.21.15.181:0/2500032334 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1efdc19640 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.475+0000 7f1efe41a640 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 -- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 msgr2=0x556d4fb1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 -- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 msgr2=0x556d4fb4c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 -- 172.21.15.181:0/2500032334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4f79a2d0 con 0x556d4fb1cd40 2024-03-29T04:09:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1efec1b640 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0089300 tx=0x7f1ef005d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1db8dbe2302ecbb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1edbfff640 1 -- 172.21.15.181:0/2500032334 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0055470 con 0x556d4fb1cd40 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1edbfff640 1 -- 172.21.15.181:0/2500032334 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0059060 con 0x556d4fb1cd40 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1edbfff640 1 -- 172.21.15.181:0/2500032334 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00614a0 con 0x556d4fb1cd40 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.476+0000 7f1f03260740 1 -- 172.21.15.181:0/2500032334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 msgr2=0x556d4fb4cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0089300 tx=0x7f1ef005d1c0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 -- 172.21.15.181:0/2500032334 shutdown_connections 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb1cd40 0x556d4fb4cb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 --2- 172.21.15.181:0/2500032334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 -- 172.21.15.181:0/2500032334 >> 172.21.15.181:0/2500032334 conn(0x556d4fb1ea70 msgr2=0x556d4fb675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 -- 172.21.15.181:0/2500032334 shutdown_connections 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 -- 172.21.15.181:0/2500032334 wait complete. 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 -- start start 2024-03-29T04:09:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.477+0000 7f1f03260740 1 Processor -- start 2024-03-29T04:09:23.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb46ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556d4fb21110 con 0x556d4fb49860 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d4fa419c0 con 0x556d4fb4e3c0 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556d4fb23950 con 0x556d4fb49c40 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1f03260740 10 client.0 _ll_register_callbacks cb 0x556d4fa25fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1efe41a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1efe41a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57678/0 (socket says 172.21.15.181:57678) 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1efe41a640 1 -- 172.21.15.181:0/445924203 learned_addr learned my addr 172.21.15.181:0/445924203 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1efdc19640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb46ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.478+0000 7f1efec1b640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 -- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 msgr2=0x556d4fb46ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb46ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 -- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb4e3c0 msgr2=0x556d4fb4b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 -- 172.21.15.181:0/445924203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4f8646b0 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efe41a640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4014da0 tx=0x7f1ef4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d123a6c2b2a722bc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4027060 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ef4010040 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0059060 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f1ef402d050 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 10 client.4425 fetch_fsmap learned FSMap version 7 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556d4fb78bb0 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0059060 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f1ef4030020 con 0x556d4fb49860 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 10 client.4425 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:09:23.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 10 client.4425 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 20 client.4425 populate_metadata read hostname 'smithi181' 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fb77fb0 0x556d4fb51f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 4 client.4425 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 --> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d4fa5a4e0 con 0x556d4fb77fb0 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.479+0000 7f1efdc19640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fb77fb0 0x556d4fb51f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.480+0000 7f1ed97fa640 10 client.4425 ms_handle_connect on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.480+0000 7f1efdc19640 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fb77fb0 0x556d4fb51f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ddc1cf904748dba server_cookie=5b19b013fa8885d3 in_seq=0 out_seq=0 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.480+0000 7f1ed97fa640 1 -- 172.21.15.181:0/445924203 <== mds.0 v2:172.21.15.187:6833/3068825067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556d4fa5a4e0 con 0x556d4fb77fb0 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.480+0000 7f1ed97fa640 10 client.4425 handle_command_reply: tid=0 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.480+0000 7f1f03260740 1 client.4425 shutdown 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 msgr2=0x556d4fb504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4014da0 tx=0x7f1ef4001250 comp rx=0 tx=0).stop 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 shutdown_connections 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fb77fb0 0x556d4fb51f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb46ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 --2- 172.21.15.181:0/445924203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb49860 0x556d4fb504b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1ed97fa640 0 client.4425 ms_handle_reset on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 >> 172.21.15.181:0/445924203 conn(0x556d4fb1cd40 msgr2=0x556d4fb207e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 shutdown_connections 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 1 -- 172.21.15.181:0/445924203 wait complete. 2024-03-29T04:09:23.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.481+0000 7f1f03260740 20 client.4425 trim_cache size 0 max 16384 2024-03-29T04:09:23.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 Processor -- start 2024-03-29T04:09:23.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 -- start start 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49860 0x556d4fb46cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81fea00 con 0x556d4fb4e3c0 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81fdeb0 con 0x556d4fb49860 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef81fe3b0 con 0x556d4fb49c40 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57686/0 (socket says 172.21.15.181:57686) 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1efec1b640 1 -- 172.21.15.181:0/1655108295 learned_addr learned my addr 172.21.15.181:0/1655108295 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.499+0000 7f1efe41a640 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49860 0x556d4fb46cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efdc19640 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 -- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 msgr2=0x556d4fb4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb4b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 -- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49860 msgr2=0x556d4fb46cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49860 0x556d4fb46cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 -- 172.21.15.181:0/1655108295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4faba020 con 0x556d4fb4e3c0 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1efec1b640 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000cdf0 tx=0x7f1ef005f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2da66a355d187e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1edb7fe640 1 -- 172.21.15.181:0/1655108295 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0087040 con 0x556d4fb4e3c0 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1edb7fe640 1 -- 172.21.15.181:0/1655108295 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0055470 con 0x556d4fb4e3c0 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1edb7fe640 1 -- 172.21.15.181:0/1655108295 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00592a0 con 0x556d4fb4e3c0 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 -- 172.21.15.181:0/1655108295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 msgr2=0x556d4fb4d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000cdf0 tx=0x7f1ef005f000 comp rx=0 tx=0).stop 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 -- 172.21.15.181:0/1655108295 shutdown_connections 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb4d7e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49c40 0x556d4fb4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 --2- 172.21.15.181:0/1655108295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49860 0x556d4fb46cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 -- 172.21.15.181:0/1655108295 >> 172.21.15.181:0/1655108295 conn(0x556d4fb1cd40 msgr2=0x556d4fb50270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.500+0000 7f1f03260740 1 -- 172.21.15.181:0/1655108295 shutdown_connections 2024-03-29T04:09:23.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.501+0000 7f1f03260740 1 -- 172.21.15.181:0/1655108295 wait complete. 2024-03-29T04:09:23.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.501+0000 7f1f03260740 1 -- start start 2024-03-29T04:09:23.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.501+0000 7f1f03260740 1 Processor -- start 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556d4fb4ea20 con 0x556d4fb4e3c0 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d4fb4ee30 con 0x556d4fb49c40 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556d4fb783f0 con 0x556d4fb49860 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1f03260740 10 client.0 _ll_register_callbacks cb 0x7f1ef4002030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efdc19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efec1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57692/0 (socket says 172.21.15.181:57692) 2024-03-29T04:09:23.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efec1b640 1 -- 172.21.15.181:0/2892260827 learned_addr learned my addr 172.21.15.181:0/2892260827 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 -- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 msgr2=0x556d4fb4dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 -- 172.21.15.181:0/2892260827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 msgr2=0x556d4fb687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efe41a640 1 -- 172.21.15.181:0/2892260827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x556d4fb4fa00 con 0x556d4fb49860 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efdc19640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.502+0000 7f1efec1b640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1efe41a640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb4d7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4001560 tx=0x7f1ef400a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07c5467432470b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3301/0 2024-03-29T04:09:23.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4036060 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mon.2 v2:172.21.15.187:3301/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ef4028930 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef0055470 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f1ef403c050 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 10 client.4331 fetch_fsmap learned FSMap version 7 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x556d4fbe4ef0 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mon.2 v2:172.21.15.187:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0055470 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mon.2 v2:172.21.15.187:3301/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f1ef403f020 con 0x556d4fb49860 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 10 client.4331 fetch_fsmap finished waiting for FSMap version 7 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 10 client.4331 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 20 client.4331 populate_metadata read hostname 'smithi181' 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fbe3ec0 0x556d4fbe42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 4 client.4331 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 --> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] -- command(tid 0: {"prefix": "status"}) -- 0x556d4fb680c0 con 0x556d4fbe3ec0 2024-03-29T04:09:23.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1efdc19640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fbe3ec0 0x556d4fbe42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.503+0000 7f1ed97fa640 10 client.4331 ms_handle_connect on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1efdc19640 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fbe3ec0 0x556d4fbe42a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2863b87ab8d49 server_cookie=1caa2d27de4ce49e in_seq=0 out_seq=0 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1ed97fa640 1 -- 172.21.15.181:0/2892260827 <== mds.0 v2:172.21.15.187:6833/3068825067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x556d4fb680c0 con 0x556d4fbe3ec0 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1ed97fa640 10 client.4331 handle_command_reply: tid=0 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 client.4331 shutdown 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 msgr2=0x556d4fb4d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb4d7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4001560 tx=0x7f1ef400a000 comp rx=0 tx=0).stop 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 shutdown_connections 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556d4fb4e3c0 0x556d4fb687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d4fb49c40 0x556d4fb4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x556d4fbe3ec0 0x556d4fbe42a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 --2- 172.21.15.181:0/2892260827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556d4fb49860 0x556d4fb4d7e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1ed97fa640 0 client.4331 ms_handle_reset on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:23.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 >> 172.21.15.181:0/2892260827 conn(0x556d4fb1cd40 msgr2=0x556d4fb46cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 shutdown_connections 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 1 -- 172.21.15.181:0/2892260827 wait complete. 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f03260740 20 client.4331 trim_cache size 0 max 16384 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "fab9a006-093b-45b9-834f-2720bd7aa224", 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4293, 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-03-29T04:09:23.491 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 7.2037780360000001, 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 7, 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 25, 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 24, 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 14.322261917000001 2024-03-29T04:09:23.492 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 msgr2=0x7f1edc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 0x7f1edc06e9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40012e0 tx=0x7f1ef4001700 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 msgr2=0x7f1ef8147200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c430 tx=0x7f1ef000c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 shutdown_connections 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80c74a0 0x7f1ef8147200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef80afd70 0x7f1ef81f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1edc06c530 0x7f1edc06e9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.504+0000 7f1f0067e640 1 --2- 172.21.15.181:0/331269790 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ef806cca0 0x7f1ef80b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.505+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 >> 172.21.15.181:0/331269790 conn(0x7f1ef8071d00 msgr2=0x7f1ef806f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.505+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 shutdown_connections 2024-03-29T04:09:23.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.505+0000 7f1f0067e640 1 -- 172.21.15.181:0/331269790 wait complete. 2024-03-29T04:09:23.495 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:09:23.495 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:23.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:09:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 Processor -- start 2024-03-29T04:09:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 -- start start 2024-03-29T04:09:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f38f415e4f0 0x7f38f40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 0x7f38f40d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38f40b9d20 con 0x7f38f40b7530 2024-03-29T04:09:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40ba000 con 0x7f38f415e4f0 2024-03-29T04:09:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f920c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38f40ba2e0 con 0x7f38f40b79c0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.617+0000 7f38f259c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f38f40b7530 0x7f38f40d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32818/0 (socket says 172.21.15.181:32818) 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f259c640 1 -- 172.21.15.181:0/204295250 learned_addr learned my addr 172.21.15.181:0/204295250 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/204295250 >> v2:172.21.15.187:3300/0 conn(0x7f38f415e4f0 0x7f38f40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f359e640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 0x7f38f40d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f1d9b640 1 -- 172.21.15.181:0/204295250 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768291035 0 0) 0x7f38f40b9d20 con 0x7f38f40b7530 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f1d9b640 1 -- 172.21.15.181:0/204295250 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38dc003520 con 0x7f38f40b7530 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f2d9d640 1 -- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 msgr2=0x7f38f40d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 0x7f38f40d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f2d9d640 1 -- 172.21.15.181:0/204295250 >> v1:172.21.15.181:6789/0 conn(0x7f38f40b7530 legacy=0x7f38f40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f2d9d640 1 -- 172.21.15.181:0/204295250 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40ceaf0 con 0x7f38f415e4f0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.618+0000 7f38f359e640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 0x7f38f40d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f40cf460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097a0 tx=0x7f38e402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cb815e3406a7fd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f1d9b640 1 -- 172.21.15.181:0/204295250 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40315a0 con 0x7f38f415e4f0 2024-03-29T04:09:23.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f1d9b640 1 -- 172.21.15.181:0/204295250 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e4037070 con 0x7f38f415e4f0 2024-03-29T04:09:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f1d9b640 1 -- 172.21.15.181:0/204295250 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4031940 con 0x7f38f415e4f0 2024-03-29T04:09:23.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 -- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 msgr2=0x7f38f40cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f40cf460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38e40097a0 tx=0x7f38e402b840 comp rx=0 tx=0).stop 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 -- 172.21.15.181:0/204295250 shutdown_connections 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b79c0 0x7f38f40d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 --2- 172.21.15.181:0/204295250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f40cf460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 -- 172.21.15.181:0/204295250 >> 172.21.15.181:0/204295250 conn(0x7f38f40676e0 msgr2=0x7f38f4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 -- 172.21.15.181:0/204295250 shutdown_connections 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.619+0000 7f38f920c640 1 -- 172.21.15.181:0/204295250 wait complete. 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.621+0000 7f38f920c640 1 Processor -- start 2024-03-29T04:09:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.621+0000 7f38f920c640 1 -- start start 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.621+0000 7f38f920c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 0x7f38f40b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f4152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415ea50 con 0x7f38f40b79c0 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38f415e8d0 con 0x7f38f415e4f0 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f38f40d30c0 con 0x7f38f40b7530 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57698/0 (socket says 172.21.15.181:57698) 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f259c640 1 -- 172.21.15.181:0/856238163 learned_addr learned my addr 172.21.15.181:0/856238163 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 0x7f38f40b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f359e640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f4152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 -- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 msgr2=0x7f38f4152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f4152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 -- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 msgr2=0x7f38f4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 -- 172.21.15.181:0/856238163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f4152fe0 con 0x7f38f40b7530 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f259c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f2d9d640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 0x7f38f40b52c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b6ff0 tx=0x7f38e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4032630 con 0x7f38f40b7530 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e40327d0 con 0x7f38f40b7530 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4032b80 con 0x7f38f40b7530 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f40b59a0 con 0x7f38f40b7530 2024-03-29T04:09:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.622+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f40b5f60 con 0x7f38f40b7530 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.623+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f4075a10 con 0x7f38f40b7530 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.624+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f38e404e430 con 0x7f38f40b7530 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.624+0000 7f38db7fe640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 0x7f38e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.624+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f38e402f030 con 0x7f38f40b7530 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.624+0000 7f38f259c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 0x7f38e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:23.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.625+0000 7f38f259c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 0x7f38e006e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38f40ce230 tx=0x7f38e8007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:23.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.629+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f38e4040070 con 0x7f38f40b7530 2024-03-29T04:09:23.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.843+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f38f415df30 con 0x7f38f40b7530 2024-03-29T04:09:23.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.844+0000 7f38db7fe640 1 -- 172.21.15.181:0/856238163 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f38e407b9d0 con 0x7f38f40b7530 2024-03-29T04:09:23.823 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:09:23.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 msgr2=0x7f38e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 0x7f38e006e9f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f38f40ce230 tx=0x7f38e8007d70 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 msgr2=0x7f38f40b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 0x7f38f40b52c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f38f40b6ff0 tx=0x7f38e40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 shutdown_connections 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38f415e4f0 0x7f38f4152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f40b79c0 0x7f38f4161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f38e006c530 0x7f38e006e9f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 --2- 172.21.15.181:0/856238163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f38f40b7530 0x7f38f40b52c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 >> 172.21.15.181:0/856238163 conn(0x7f38f40676e0 msgr2=0x7f38f4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 shutdown_connections 2024-03-29T04:09:23.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23.846+0000 7f38f920c640 1 -- 172.21.15.181:0/856238163 wait complete. 2024-03-29T04:09:23.836 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:09:23.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:09:23.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-03-29T04:09:23.836 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-29T04:09:23.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:23.837 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:09:23.837 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:09:23.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:09:23.837 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:09:23.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:23.837 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-03-29T04:09:23.852 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:23.852 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:23.852 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:23.852 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:09:23.853 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42733sec preferred_lft 42733sec 2024-03-29T04:09:23.854 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-03-29T04:09:23.854 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:23.854 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:09:23.854 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:23.854 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:23.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:09:23.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2024-03-29T04:09:23.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2024-03-29T04:09:23.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:09:23.855 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:23.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:24.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:24.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:24.057 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:09:24.084 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-03-29T04:09:24.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:24.085 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-03-29T04:09:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-03-29T04:09:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:09:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:09:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:09:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:09:24.155 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:24.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:24.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:24.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:24.501 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:09:24.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:24.516 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-03-29T04:09:24.572 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:24.572 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:24.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:24.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-29T04:09:24.572 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:24.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:24.695 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-29T04:09:24.696 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:24.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:24.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:24.861 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:24.861 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:24.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-03-29T04:09:24.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:09:24.862 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:24.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:24.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:24.943 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-29T04:09:24.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:24.943 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:24.970 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T04:09:24.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:24.970 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:25.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-03-29T04:09:25.092 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-03-29T04:09:25.148 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/selinux 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-03-29T04:09:25.149 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:25.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:25.150 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-03-29T04:09:25.203 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T04:09:25.203 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-29T04:09:25.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-03-29T04:09:25.273 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-03-29T04:09:25.329 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/selinux 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysctl.service 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/nfs/rpc_pipefs 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-03-29T04:09:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-0 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-1 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-2 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-3 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:25.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:25.331 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-03-29T04:09:25.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi181.stderr:2024-03-29T04:09:25.355+0000 7fa42ee47480 -1 init, newargv = 0x5646e4aa3ed0 newargc=15 2024-03-29T04:09:25.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi181.stderr:ceph-fuse[44712]: starting ceph client 2024-03-29T04:09:25.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi181.stderr:ceph-fuse[44712]: starting fuse 2024-03-29T04:09:25.350 INFO:teuthology.orchestra.run.smithi181.stdout:41 2024-03-29T04:09:25.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-03-29T04:09:25.351 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-03-29T04:09:25.351 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-03-29T04:09:25.351 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-03-29T04:09:25.351 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-29T04:09:25.352 DEBUG:teuthology.orchestra.run.smithi181:> 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-03-29T04:09:25.353 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-03-29T04:09:25.354 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T04:09:25.355 DEBUG:teuthology.orchestra.run.smithi181:> 2024-03-29T04:09:25.355 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.admin")) 2024-03-29T04:09:25.355 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-03-29T04:09:25.472 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.admin.44712.asok 2024-03-29T04:09:25.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:25.483 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44712.asok 2024-03-29T04:09:25.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:25.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44712.asok status 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "69e2cd73c52284dfd5096c66b60c6e6b26a3c447", 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.0.0-2520-g69e2cd73 (69e2cd73c52284dfd5096c66b60c6e6b26a3c447) squid (dev)", 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "admin", 2024-03-29T04:09:25.619 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "44712", 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4446, 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 4446 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-03-29T04:09:25.620 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3147268729 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3147268729 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.4446 192.168.0.1:0/3147268729", 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/3147268729", 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 7, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 25, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-03-29T04:09:25.621 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-03-29T04:09:25.622 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:09:25.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:25.627 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-03-29T04:09:25.627 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:25.642 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-03-29T04:09:25.642 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:25.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:25.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:25.709 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-29T04:09:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:25.811 INFO:teuthology.orchestra.run.smithi181.stdout:# file: . 2024-03-29T04:09:25.811 INFO:teuthology.orchestra.run.smithi181.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-29T04:09:25.811 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:25.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:25.817 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:25.817 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:25.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.995+0000 7f0ddb2eb640 1 Processor -- start 2024-03-29T04:09:25.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.995+0000 7f0ddb2eb640 1 -- start start 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0ddb2eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 0x7f0dd4160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0ddb2eb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0dd4160660 0x7f0dd40ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0ddb2eb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd4146040 con 0x7f0dd4145c60 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0ddb2eb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd4160e50 con 0x7f0dd4160660 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0ddb2eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd415d8b0 con 0x7f0dd4071c30 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0dd9087640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0dd4145c60 0x7f0dd4071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32820/0 (socket says 172.21.15.181:32820) 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0dd9087640 1 -- 172.21.15.181:0/1046917010 learned_addr learned my addr 172.21.15.181:0/1046917010 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.996+0000 7f0dd9888640 1 --2- 172.21.15.181:0/1046917010 >> v2:172.21.15.181:3300/0 conn(0x7f0dd4160660 0x7f0dd40ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd8886640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 0x7f0dd4160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd9888640 1 -- 172.21.15.181:0/1046917010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 msgr2=0x7f0dd4160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd9888640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 0x7f0dd4160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd9888640 1 -- 172.21.15.181:0/1046917010 >> v1:172.21.15.181:6789/0 conn(0x7f0dd4145c60 legacy=0x7f0dd4071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd9888640 1 -- 172.21.15.181:0/1046917010 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd415caf0 con 0x7f0dd4160660 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dcbfff640 1 -- 172.21.15.181:0/1046917010 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502005851 0 0) 0x7f0dd4146040 con 0x7f0dd4145c60 2024-03-29T04:09:25.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.997+0000 7f0dd8886640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 0x7f0dd4160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0dd9888640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd40ca830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0097a0 tx=0x7f0dcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a32730a23745f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0dcbfff640 1 -- 172.21.15.181:0/1046917010 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0315a0 con 0x7f0dd4160660 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0dcbfff640 1 -- 172.21.15.181:0/1046917010 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc037070 con 0x7f0dd4160660 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0dcbfff640 1 -- 172.21.15.181:0/1046917010 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc031960 con 0x7f0dd4160660 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/1046917010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 msgr2=0x7f0dd40ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd40ca830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0097a0 tx=0x7f0dcc02b840 comp rx=0 tx=0).stop 2024-03-29T04:09:25.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/1046917010 shutdown_connections 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd40ca830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/1046917010 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4071c30 0x7f0dd4160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/1046917010 >> 172.21.15.181:0/1046917010 conn(0x7f0dd40b1c90 msgr2=0x7f0dd40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/1046917010 shutdown_connections 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:25.998+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/1046917010 wait complete. 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 Processor -- start 2024-03-29T04:09:25.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 -- start start 2024-03-29T04:09:25.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4071c30 0x7f0dd415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4145c60 0x7f0dd41442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4160b10 con 0x7f0dd4160660 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd415d0b0 con 0x7f0dd4071c30 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0ddb2eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd40cad70 con 0x7f0dd4145c60 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0dd9888640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0dd9888640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57706/0 (socket says 172.21.15.181:57706) 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0dd9888640 1 -- 172.21.15.181:0/2831308998 learned_addr learned my addr 172.21.15.181:0/2831308998 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.000+0000 7f0dd9087640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4071c30 0x7f0dd415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd8886640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4145c60 0x7f0dd41442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 -- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4145c60 msgr2=0x7f0dd41442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4145c60 0x7f0dd41442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 -- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4071c30 msgr2=0x7f0dd415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:25.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4071c30 0x7f0dd415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 -- 172.21.15.181:0/2831308998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd41424b0 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dd9888640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc034040 tx=0x7f0dcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc032630 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0327f0 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd4142790 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc032be0 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.001+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd412d3b0 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.002+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd4075a10 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.003+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0dcc03b3f0 con 0x7f0dd4160660 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.003+0000 7f0dc9ffb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 0x7f0db806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.003+0000 7f0dd9087640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 0x7f0db806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:25.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.003+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0dcc02f030 con 0x7f0dd4160660 2024-03-29T04:09:25.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.004+0000 7f0dd9087640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 0x7f0db806e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00012e0 tx=0x7f0dd0001700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:25.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.008+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0dcc040070 con 0x7f0dd4160660 2024-03-29T04:09:26.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.181+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dd40ca610 con 0x7f0dd4160660 2024-03-29T04:09:26.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.183+0000 7f0dc9ffb640 1 -- 172.21.15.181:0/2831308998 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f0dcc07b360 con 0x7f0dd4160660 2024-03-29T04:09:26.163 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:26.163 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:26.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 msgr2=0x7f0db806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 0x7f0db806e9f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00012e0 tx=0x7f0dd0001700 comp rx=0 tx=0).stop 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 msgr2=0x7f0dd41447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc034040 tx=0x7f0dcc0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 shutdown_connections 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd4160660 0x7f0dd41447e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0db806c530 0x7f0db806e9f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0dd4145c60 0x7f0dd41442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 --2- 172.21.15.181:0/2831308998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0dd4071c30 0x7f0dd415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 >> 172.21.15.181:0/2831308998 conn(0x7f0dd40b1c90 msgr2=0x7f0dd40b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 shutdown_connections 2024-03-29T04:09:26.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.185+0000 7f0ddb2eb640 1 -- 172.21.15.181:0/2831308998 wait complete. 2024-03-29T04:09:26.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:26.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.299+0000 7f91f54d9640 1 Processor -- start 2024-03-29T04:09:26.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.299+0000 7f91f54d9640 1 -- start start 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91f54d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91f54d9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 0x7f91f015d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91f54d9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f00cc970 con 0x7f91f00ce490 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91f54d9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f91f00ce8f0 con 0x7f91f015ceb0 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91f54d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f91f0146150 con 0x7f91f0145d70 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91ef7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 0x7f91f015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91ee7fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f91f00ce490 0x7f91f00cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32822/0 (socket says 172.21.15.181:32822) 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91ee7fc640 1 -- 172.21.15.181:0/2001290390 learned_addr learned my addr 172.21.15.181:0/2001290390 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:26.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91ef7fe640 1 --2- 172.21.15.181:0/2001290390 >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 0x7f91f015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57710/0 (socket says 172.21.15.181:57710) 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.300+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2001290390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.301+0000 7f91eeffd640 1 -- 172.21.15.181:0/2001290390 >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 msgr2=0x7f91f015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.301+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2001290390 >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 0x7f91f015d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.301+0000 7f91eeffd640 1 -- 172.21.15.181:0/2001290390 >> v1:172.21.15.181:6789/0 conn(0x7f91f00ce490 legacy=0x7f91f00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.301+0000 7f91eeffd640 1 -- 172.21.15.181:0/2001290390 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f00ccc50 con 0x7f91f0145d70 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.301+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2001290390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0160880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009d90 tx=0x7f91e002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52bdf4af1f5476b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91edffb640 1 -- 172.21.15.181:0/2001290390 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0039070 con 0x7f91f0145d70 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91edffb640 1 -- 172.21.15.181:0/2001290390 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e002cbc0 con 0x7f91f0145d70 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91edffb640 1 -- 172.21.15.181:0/2001290390 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0034440 con 0x7f91f0145d70 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 -- 172.21.15.181:0/2001290390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 msgr2=0x7f91f0160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2001290390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0160880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009d90 tx=0x7f91e002be10 comp rx=0 tx=0).stop 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 -- 172.21.15.181:0/2001290390 shutdown_connections 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2001290390 >> v2:172.21.15.181:3300/0 conn(0x7f91f015ceb0 0x7f91f015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2001290390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0160880 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 -- 172.21.15.181:0/2001290390 >> 172.21.15.181:0/2001290390 conn(0x7f91f00b3e70 msgr2=0x7f91f00b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 -- 172.21.15.181:0/2001290390 shutdown_connections 2024-03-29T04:09:26.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.302+0000 7f91f54d9640 1 -- 172.21.15.181:0/2001290390 wait complete. 2024-03-29T04:09:26.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.303+0000 7f91f54d9640 1 Processor -- start 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 -- start start 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f00ce490 0x7f91f006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f91f00cc500 con 0x7f91f015ceb0 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91f015ca20 con 0x7f91f00ce490 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91f54d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f91f00ccf30 con 0x7f91f0145d70 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57716/0 (socket says 172.21.15.181:57716) 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91ef7fe640 1 -- 172.21.15.181:0/2975812075 learned_addr learned my addr 172.21.15.181:0/2975812075 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f00ce490 0x7f91f006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.304+0000 7f91ee7fc640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 -- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 msgr2=0x7f91f0140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 -- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f00ce490 msgr2=0x7f91f006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f00ce490 0x7f91f006e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 -- 172.21.15.181:0/2975812075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f015b970 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91ef7fe640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f91e800e930 tx=0x7f91e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8017070 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e800c980 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f015e490 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800cd70 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.305+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f015ea50 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.306+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f0075890 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.306+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f91e801a070 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.307+0000 7f91d77fe640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 0x7f91dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.307+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f91e808bc40 con 0x7f91f015ceb0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.307+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 0x7f91dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.307+0000 7f91eeffd640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 0x7f91dc06ea20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91e0036000 tx=0x7f91e0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.311+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f91e8059a20 con 0x7f91f015ceb0 2024-03-29T04:09:26.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.486+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91f0068f00 con 0x7f91f015ceb0 2024-03-29T04:09:26.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.486+0000 7f91d77fe640 1 -- 172.21.15.181:0/2975812075 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f91e805dd20 con 0x7f91f015ceb0 2024-03-29T04:09:26.466 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 msgr2=0x7f91dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 0x7f91dc06ea20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91e0036000 tx=0x7f91e0030000 comp rx=0 tx=0).stop 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 msgr2=0x7f91f0156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f91e800e930 tx=0x7f91e800edf0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 shutdown_connections 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f015ceb0 0x7f91f0156f40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f91dc06c560 0x7f91dc06ea20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91f0145d70 0x7f91f0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 --2- 172.21.15.181:0/2975812075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91f00ce490 0x7f91f006e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 >> 172.21.15.181:0/2975812075 conn(0x7f91f00b3e70 msgr2=0x7f91f00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 shutdown_connections 2024-03-29T04:09:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.488+0000 7f91f54d9640 1 -- 172.21.15.181:0/2975812075 wait complete. 2024-03-29T04:09:26.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:26.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.605+0000 7fb461b8d640 1 Processor -- start 2024-03-29T04:09:26.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.605+0000 7fb461b8d640 1 -- start start 2024-03-29T04:09:26.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb461b8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c15f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb461b8d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb45c162070 0x7fb45c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb461b8d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c162610 con 0x7fb45c0b3c90 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb461b8d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb45c06fac0 con 0x7fb45c162070 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb461b8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c0b40d0 con 0x7fb45c0afd60 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45affd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb45c162070 0x7fb45c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45bfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb45c0b3c90 0x7fb45c06e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32824/0 (socket says 172.21.15.181:32824) 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45bfff640 1 -- 172.21.15.181:0/870869458 learned_addr learned my addr 172.21.15.181:0/870869458 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45b7fe640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45affd640 1 -- 172.21.15.181:0/870869458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 msgr2=0x7fb45c15f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45affd640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c15f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45affd640 1 -- 172.21.15.181:0/870869458 >> v1:172.21.15.181:6789/0 conn(0x7fb45c0b3c90 legacy=0x7fb45c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45affd640 1 -- 172.21.15.181:0/870869458 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c1628f0 con 0x7fb45c162070 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45a7fc640 1 -- 172.21.15.181:0/870869458 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17938296 0 0) 0x7fb45c162610 con 0x7fb45c0b3c90 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.606+0000 7fb45b7fe640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c15f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:26.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.607+0000 7fb45affd640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c0b3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4500097a0 tx=0x7fb45002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a350fee3a7f0992 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.607+0000 7fb45a7fc640 1 -- 172.21.15.181:0/870869458 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450035020 con 0x7fb45c162070 2024-03-29T04:09:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.607+0000 7fb45a7fc640 1 -- 172.21.15.181:0/870869458 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb450031830 con 0x7fb45c162070 2024-03-29T04:09:26.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.607+0000 7fb45a7fc640 1 -- 172.21.15.181:0/870869458 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450031c00 con 0x7fb45c162070 2024-03-29T04:09:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 -- 172.21.15.181:0/870869458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 msgr2=0x7fb45c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c0b3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4500097a0 tx=0x7fb45002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 -- 172.21.15.181:0/870869458 shutdown_connections 2024-03-29T04:09:26.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c0b3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 --2- 172.21.15.181:0/870869458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c15f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 -- 172.21.15.181:0/870869458 >> 172.21.15.181:0/870869458 conn(0x7fb45c0d0840 msgr2=0x7fb45c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 -- 172.21.15.181:0/870869458 shutdown_connections 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.608+0000 7fb461b8d640 1 -- 172.21.15.181:0/870869458 wait complete. 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.609+0000 7fb461b8d640 1 Processor -- start 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.609+0000 7fb461b8d640 1 -- start start 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45c0b3c90 0x7fb45c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c162eb0 con 0x7fb45c162070 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0b0140 con 0x7fb45c0b3c90 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c06cb10 con 0x7fb45c0afd60 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57734/0 (socket says 172.21.15.181:57734) 2024-03-29T04:09:26.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 -- 172.21.15.181:0/290099255 learned_addr learned my addr 172.21.15.181:0/290099255 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45affd640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45c0b3c90 0x7fb45c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45b7fe640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 -- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 msgr2=0x7fb45c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 -- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45c0b3c90 msgr2=0x7fb45c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45c0b3c90 0x7fb45c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 -- 172.21.15.181:0/290099255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c150580 con 0x7fb45c162070 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb45bfff640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c7a0 tx=0x7fb45400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454018070 con 0x7fb45c162070 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45400a640 con 0x7fb45c162070 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c150860 con 0x7fb45c162070 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400aab0 con 0x7fb45c162070 2024-03-29T04:09:26.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.610+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c0cd090 con 0x7fb45c162070 2024-03-29T04:09:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.611+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c150a30 con 0x7fb45c162070 2024-03-29T04:09:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.612+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb45401b070 con 0x7fb45c162070 2024-03-29T04:09:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.612+0000 7fb443fff640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 0x7fb44806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.612+0000 7fb45b7fe640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 0x7fb44806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.593 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.612+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb45408af20 con 0x7fb45c162070 2024-03-29T04:09:26.594 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.613+0000 7fb45b7fe640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 0x7fb44806e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009560 tx=0x7fb44c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.617+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb454058c40 con 0x7fb45c162070 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.791+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb45c075890 con 0x7fb45c162070 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.791+0000 7fb443fff640 1 -- 172.21.15.181:0/290099255 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fb45405cf40 con 0x7fb45c162070 2024-03-29T04:09:26.771 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:26.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 msgr2=0x7fb44806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 0x7fb44806e9f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009560 tx=0x7fb44c009190 comp rx=0 tx=0).stop 2024-03-29T04:09:26.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 msgr2=0x7fb45c14efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c7a0 tx=0x7fb45400cc60 comp rx=0 tx=0).stop 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 shutdown_connections 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb45c162070 0x7fb45c14efb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb45c0b3c90 0x7fb45c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb44806c530 0x7fb44806e9f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 --2- 172.21.15.181:0/290099255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb45c0afd60 0x7fb45c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.793+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 >> 172.21.15.181:0/290099255 conn(0x7fb45c0d0840 msgr2=0x7fb45c06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.794+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 shutdown_connections 2024-03-29T04:09:26.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.794+0000 7fb461b8d640 1 -- 172.21.15.181:0/290099255 wait complete. 2024-03-29T04:09:26.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:26.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.907+0000 7f79fb1ef640 1 Processor -- start 2024-03-29T04:09:26.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.907+0000 7f79fb1ef640 1 -- start start 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40afd60 0x7f79f415f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 0x7f79f40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40b3c90 0x7f79f406e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f79f406fac0 con 0x7f79f40afd60 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79f4162610 con 0x7f79f4162070 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79fb1ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79f41628f0 con 0x7f79f40b3c90 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40afd60 0x7f79f415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40afd60 0x7f79f415f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57748/0 (socket says 172.21.15.181:57748) 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 0x7f79f40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 -- 172.21.15.181:0/3872207857 learned_addr learned my addr 172.21.15.181:0/3872207857 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f978c640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40b3c90 0x7f79f406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 -- 172.21.15.181:0/3872207857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40b3c90 msgr2=0x7f79f406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40b3c90 0x7f79f406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 -- 172.21.15.181:0/3872207857 >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 msgr2=0x7f79f40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3872207857 >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 0x7f79f40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f8f8b640 1 -- 172.21.15.181:0/3872207857 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f4162bd0 con 0x7f79f40afd60 2024-03-29T04:09:26.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.908+0000 7f79f3fff640 1 --2- 172.21.15.181:0/3872207857 >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 0x7f79f40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40afd60 0x7f79f415f500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79f406d060 tx=0x7f79e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f2bf4d28478c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79f37fe640 1 -- 172.21.15.181:0/3872207857 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8035020 con 0x7f79f40afd60 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79f37fe640 1 -- 172.21.15.181:0/3872207857 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8031870 con 0x7f79f40afd60 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79f37fe640 1 -- 172.21.15.181:0/3872207857 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8031c40 con 0x7f79f40afd60 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3872207857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40afd60 msgr2=0x7f79f415f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40afd60 0x7f79f415f500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79f406d060 tx=0x7f79e802b910 comp rx=0 tx=0).stop 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3872207857 shutdown_connections 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40b3c90 0x7f79f406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3872207857 >> v2:172.21.15.187:3300/0 conn(0x7f79f4162070 0x7f79f40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3872207857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40afd60 0x7f79f415f500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.909+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3872207857 >> 172.21.15.181:0/3872207857 conn(0x7f79f40d0840 msgr2=0x7f79f40d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.910+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3872207857 shutdown_connections 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.910+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3872207857 wait complete. 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.911+0000 7f79fb1ef640 1 Processor -- start 2024-03-29T04:09:26.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.911+0000 7f79fb1ef640 1 -- start start 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.911+0000 7f79fb1ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40afd60 0x7f79f41539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.911+0000 7f79fb1ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40b3c90 0x7f79f4153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.911+0000 7f79fb1ef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79fb1ef640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f79f406e5d0 con 0x7f79f4162070 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79fb1ef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79f4162eb0 con 0x7f79f40b3c90 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79fb1ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79f406e750 con 0x7f79f40afd60 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57752/0 (socket says 172.21.15.181:57752) 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 -- 172.21.15.181:0/3948301007 learned_addr learned my addr 172.21.15.181:0/3948301007 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40afd60 0x7f79f41539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f3fff640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40b3c90 0x7f79f4153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 -- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40afd60 msgr2=0x7f79f41539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40afd60 0x7f79f41539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 -- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40b3c90 msgr2=0x7f79f4153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40b3c90 0x7f79f4153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 -- 172.21.15.181:0/3948301007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f41e80a0 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f978c640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00ef00 tx=0x7f79ec00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec018020 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec014400 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f41e8380 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0147f0 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.912+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f414b980 con 0x7f79f4162070 2024-03-29T04:09:26.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.913+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4075890 con 0x7f79f4162070 2024-03-29T04:09:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.914+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f79ec01b020 con 0x7f79f4162070 2024-03-29T04:09:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.914+0000 7f79f17fa640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.914+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.914+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f79ec08c3f0 con 0x7f79f4162070 2024-03-29T04:09:26.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.915+0000 7f79f8f8b640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:26.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:26.919+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f79ec022020 con 0x7f79f4162070 2024-03-29T04:09:27.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.092+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79f4158e20 con 0x7f79f4162070 2024-03-29T04:09:27.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.092+0000 7f79f17fa640 1 -- 172.21.15.181:0/3948301007 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f79ec059ff0 con 0x7f79f4162070 2024-03-29T04:09:27.072 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 msgr2=0x7f79d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).stop 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 msgr2=0x7f79f41f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00ef00 tx=0x7f79ec00b480 comp rx=0 tx=0).stop 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 shutdown_connections 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f4162070 0x7f79f41f4540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40b3c90 0x7f79f4153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.074 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 --2- 172.21.15.181:0/3948301007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f40afd60 0x7f79f41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 >> 172.21.15.181:0/3948301007 conn(0x7f79f40d0840 msgr2=0x7f79f40ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 shutdown_connections 2024-03-29T04:09:27.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.094+0000 7f79fb1ef640 1 -- 172.21.15.181:0/3948301007 wait complete. 2024-03-29T04:09:27.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:27.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.208+0000 7fa6388c8640 1 Processor -- start 2024-03-29T04:09:27.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.208+0000 7fa6388c8640 1 -- start start 2024-03-29T04:09:27.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa6388c8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa63415eba0 0x7fa63406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa6388c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 0x7fa6340cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa6388c8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340d1170 con 0x7fa63415e7c0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa6388c8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340cc840 con 0x7fa63415eba0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa6388c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340d1450 con 0x7fa6340b4800 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa631d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa63415eba0 0x7fa63406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa631d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa63415eba0 0x7fa63406cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57764/0 (socket says 172.21.15.181:57764) 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa63259c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa63415e7c0 0x7fa63406c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32838/0 (socket says 172.21.15.181:32838) 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa631d9b640 1 -- 172.21.15.181:0/758077266 learned_addr learned my addr 172.21.15.181:0/758077266 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.209+0000 7fa632d9d640 1 --2- 172.21.15.181:0/758077266 >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 0x7fa6340cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa631d9b640 1 -- 172.21.15.181:0/758077266 >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 msgr2=0x7fa6340cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa631d9b640 1 --2- 172.21.15.181:0/758077266 >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 0x7fa6340cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa631d9b640 1 -- 172.21.15.181:0/758077266 >> v1:172.21.15.181:6789/0 conn(0x7fa63415e7c0 legacy=0x7fa63406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa631d9b640 1 -- 172.21.15.181:0/758077266 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340d1730 con 0x7fa63415eba0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa63159a640 1 -- 172.21.15.181:0/758077266 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097277812 0 0) 0x7fa6340d1170 con 0x7fa63415e7c0 2024-03-29T04:09:27.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa632d9d640 1 --2- 172.21.15.181:0/758077266 >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 0x7fa6340cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa631d9b640 1 --2- 172.21.15.181:0/758077266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415eba0 0x7fa63406cf70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa6340d01c0 tx=0x7fa62802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb745e7ebc1a3cc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.210+0000 7fa63159a640 1 -- 172.21.15.181:0/758077266 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628034070 con 0x7fa63415eba0 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa63159a640 1 -- 172.21.15.181:0/758077266 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa628030860 con 0x7fa63415eba0 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa63159a640 1 -- 172.21.15.181:0/758077266 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628030c30 con 0x7fa63415eba0 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 -- 172.21.15.181:0/758077266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415eba0 msgr2=0x7fa63406cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 --2- 172.21.15.181:0/758077266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415eba0 0x7fa63406cf70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa6340d01c0 tx=0x7fa62802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 -- 172.21.15.181:0/758077266 shutdown_connections 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 --2- 172.21.15.181:0/758077266 >> v2:172.21.15.187:3300/0 conn(0x7fa6340b4800 0x7fa6340cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 --2- 172.21.15.181:0/758077266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415eba0 0x7fa63406cf70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 -- 172.21.15.181:0/758077266 >> 172.21.15.181:0/758077266 conn(0x7fa6340b3ce0 msgr2=0x7fa6340b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 -- 172.21.15.181:0/758077266 shutdown_connections 2024-03-29T04:09:27.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.211+0000 7fa6388c8640 1 -- 172.21.15.181:0/758077266 wait complete. 2024-03-29T04:09:27.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.212+0000 7fa6388c8640 1 Processor -- start 2024-03-29T04:09:27.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.212+0000 7fa6388c8640 1 -- start start 2024-03-29T04:09:27.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b4800 0x7fa634156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa63415eba0 0x7fa6340c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406d630 con 0x7fa63415e7c0 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa63406d4b0 con 0x7fa6340b4800 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa6388c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6340cd210 con 0x7fa63415eba0 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa631d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa631d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57776/0 (socket says 172.21.15.181:57776) 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa631d9b640 1 -- 172.21.15.181:0/2676162772 learned_addr learned my addr 172.21.15.181:0/2676162772 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa632d9d640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa63415eba0 0x7fa6340c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.213+0000 7fa63259c640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b4800 0x7fa634156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 -- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa63415eba0 msgr2=0x7fa6340c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa63415eba0 0x7fa6340c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 -- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b4800 msgr2=0x7fa634156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b4800 0x7fa634156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 -- 172.21.15.181:0/2676162772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341f7ab0 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa631d9b640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa62802b9e0 tx=0x7fa6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628034040 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62803ada0 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341f7d90 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628044630 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.214+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341f8350 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.215+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa634075890 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.215+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa62804b060 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.216+0000 7fa622ffd640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 0x7fa61406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.216+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa6280ad980 con 0x7fa63415e7c0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.216+0000 7fa63259c640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 0x7fa61406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.216+0000 7fa63259c640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 0x7fa61406ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6240012e0 tx=0x7fa624001700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.220+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa62803f070 con 0x7fa63415e7c0 2024-03-29T04:09:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.398+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa6340c5f00 con 0x7fa63415e7c0 2024-03-29T04:09:27.378 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.399+0000 7fa622ffd640 1 -- 172.21.15.181:0/2676162772 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fa62807bc60 con 0x7fa63415e7c0 2024-03-29T04:09:27.379 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:27.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 msgr2=0x7fa61406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 0x7fa61406ea20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6240012e0 tx=0x7fa624001700 comp rx=0 tx=0).stop 2024-03-29T04:09:27.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 msgr2=0x7fa6340c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa62802b9e0 tx=0x7fa6280046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 shutdown_connections 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa61406c560 0x7fa61406ea20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa63415eba0 0x7fa6340c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa63415e7c0 0x7fa6340c8270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 --2- 172.21.15.181:0/2676162772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6340b4800 0x7fa634156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 >> 172.21.15.181:0/2676162772 conn(0x7fa6340b3ce0 msgr2=0x7fa63415c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 shutdown_connections 2024-03-29T04:09:27.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.401+0000 7fa6388c8640 1 -- 172.21.15.181:0/2676162772 wait complete. 2024-03-29T04:09:27.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:09:27.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 Processor -- start 2024-03-29T04:09:27.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 -- start start 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4a415f890 0x7fe4a4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a415feb0 con 0x7fe4a40cb050 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a415c910 con 0x7fe4a40cbed0 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.513+0000 7fe4aac2a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a415c480 con 0x7fe4a415f890 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a91c7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4a415f890 0x7fe4a4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a89c6640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe4a40cb050 0x7fe4a40cb7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:32842/0 (socket says 172.21.15.181:32842) 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a91c7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4a415f890 0x7fe4a4151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57790/0 (socket says 172.21.15.181:57790) 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a89c6640 1 -- 172.21.15.181:0/1724606602 learned_addr learned my addr 172.21.15.181:0/1724606602 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a91c7640 1 -- 172.21.15.181:0/1724606602 >> v1:172.21.15.187:6789/0 conn(0x7fe4a40cbed0 legacy=0x7fe4a415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a91c7640 1 -- 172.21.15.181:0/1724606602 >> v1:172.21.15.181:6789/0 conn(0x7fe4a40cb050 legacy=0x7fe4a40cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a91c7640 1 -- 172.21.15.181:0/1724606602 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a415bb50 con 0x7fe4a415f890 2024-03-29T04:09:27.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.514+0000 7fe4a37fe640 1 -- 172.21.15.181:0/1724606602 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187237008 0 0) 0x7fe4a415feb0 con 0x7fe4a40cb050 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4a91c7640 1 --2- 172.21.15.181:0/1724606602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a4151c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009d90 tx=0x7fe49c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dbdf2e60c6c928 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4a37fe640 1 -- 172.21.15.181:0/1724606602 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c039070 con 0x7fe4a415f890 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4a37fe640 1 -- 172.21.15.181:0/1724606602 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c02cd00 con 0x7fe4a415f890 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4a37fe640 1 -- 172.21.15.181:0/1724606602 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c034660 con 0x7fe4a415f890 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 -- 172.21.15.181:0/1724606602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 msgr2=0x7fe4a4151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/1724606602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a4151c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009d90 tx=0x7fe49c02bf50 comp rx=0 tx=0).stop 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 -- 172.21.15.181:0/1724606602 shutdown_connections 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/1724606602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a4151c20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 -- 172.21.15.181:0/1724606602 >> 172.21.15.181:0/1724606602 conn(0x7fe4a406d610 msgr2=0x7fe4a406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 -- 172.21.15.181:0/1724606602 shutdown_connections 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.515+0000 7fe4aac2a640 1 -- 172.21.15.181:0/1724606602 wait complete. 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.516+0000 7fe4aac2a640 1 Processor -- start 2024-03-29T04:09:27.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.516+0000 7fe4aac2a640 1 -- start start 2024-03-29T04:09:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a40cb050 0x7fe4a40af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a40cbed0 0x7fe4a40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40d08b0 con 0x7fe4a415f890 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a415c110 con 0x7fe4a40cbed0 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4aac2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a40cb430 con 0x7fe4a40cb050 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4a91c7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4a91c7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:57798/0 (socket says 172.21.15.181:57798) 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4a91c7640 1 -- 172.21.15.181:0/828638402 learned_addr learned my addr 172.21.15.181:0/828638402 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4a3fff640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a40cbed0 0x7fe4a40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.517+0000 7fe4a89c6640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a40cb050 0x7fe4a40af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 -- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a40cb050 msgr2=0x7fe4a40af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a40cb050 0x7fe4a40af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 -- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a40cbed0 msgr2=0x7fe4a40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a40cbed0 0x7fe4a40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 -- 172.21.15.181:0/828638402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a40b3010 con 0x7fe4a415f890 2024-03-29T04:09:27.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a91c7640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009d90 tx=0x7fe49c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c039050 con 0x7fe4a415f890 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c03a040 con 0x7fe4a415f890 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a40c69b0 con 0x7fe4a415f890 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c03e610 con 0x7fe4a415f890 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.518+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a40ae220 con 0x7fe4a415f890 2024-03-29T04:09:27.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.519+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a4075900 con 0x7fe4a415f890 2024-03-29T04:09:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.519+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe49c047030 con 0x7fe4a415f890 2024-03-29T04:09:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.520+0000 7fe4a17fa640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 0x7fe48806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.520+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe49c0ad940 con 0x7fe4a415f890 2024-03-29T04:09:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.520+0000 7fe4a89c6640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 0x7fe48806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.520+0000 7fe4a89c6640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 0x7fe48806ea20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe498000f60 tx=0x7fe498007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.524+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4a40b3010 con 0x7fe4a415f890 2024-03-29T04:09:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.711+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4a406f6b0 con 0x7fe4a415f890 2024-03-29T04:09:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.712+0000 7fe4a17fa640 1 -- 172.21.15.181:0/828638402 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fe49c07b720 con 0x7fe4a415f890 2024-03-29T04:09:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:09:27.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.713+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 msgr2=0x7fe48806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.713+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 0x7fe48806ea20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe498000f60 tx=0x7fe498007e40 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.713+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 msgr2=0x7fe4a40c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.713+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009d90 tx=0x7fe49c004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 shutdown_connections 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4a415f890 0x7fe4a40c6470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a40cbed0 0x7fe4a40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe48806c560 0x7fe48806ea20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 --2- 172.21.15.181:0/828638402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a40cb050 0x7fe4a40af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 >> 172.21.15.181:0/828638402 conn(0x7fe4a406d610 msgr2=0x7fe4a40b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 shutdown_connections 2024-03-29T04:09:27.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.714+0000 7fe4aac2a640 1 -- 172.21.15.181:0/828638402 wait complete. 2024-03-29T04:09:27.704 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:27.704 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:27.719 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:27.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.880+0000 7fa058559640 1 Processor -- start 2024-03-29T04:09:27.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.880+0000 7fa058559640 1 -- start start 2024-03-29T04:09:27.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.880+0000 7fa058559640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa050155990 0x7fa05006c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.880+0000 7fa058559640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa0500b4050 0x7fa0500cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa058559640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa05015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa058559640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa0501622a0 con 0x7fa050155990 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa058559640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa0501624f0 con 0x7fa0500b4050 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa058559640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0501627d0 con 0x7fa0500d0300 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa050155990 0x7fa05006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa050155990 0x7fa05006c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34126/0 (socket says 172.21.15.181:34126) 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 -- 172.21.15.181:0/708962912 learned_addr learned my addr 172.21.15.181:0/708962912 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa056af6640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa05015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa055af4640 1 --2- 172.21.15.181:0/708962912 >> v2:172.21.15.187:3300/0 conn(0x7fa0500b4050 0x7fa0500cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 -- 172.21.15.181:0/708962912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 msgr2=0x7fa05015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa05015eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 -- 172.21.15.181:0/708962912 >> v2:172.21.15.187:3300/0 conn(0x7fa0500b4050 msgr2=0x7fa0500cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 --2- 172.21.15.181:0/708962912 >> v2:172.21.15.187:3300/0 conn(0x7fa0500b4050 0x7fa0500cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa0562f5640 1 -- 172.21.15.181:0/708962912 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa050162ab0 con 0x7fa050155990 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.881+0000 7fa056af6640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa05015eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa0562f5640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa05006c690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0500b4ae0 tx=0x7fa04002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe32c7f07701b3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa0552f3640 1 -- 172.21.15.181:0/708962912 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04002e070 con 0x7fa050155990 2024-03-29T04:09:27.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa0552f3640 1 -- 172.21.15.181:0/708962912 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa040037070 con 0x7fa050155990 2024-03-29T04:09:27.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa0552f3640 1 -- 172.21.15.181:0/708962912 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040031a00 con 0x7fa050155990 2024-03-29T04:09:27.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 -- 172.21.15.181:0/708962912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 msgr2=0x7fa05006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa05006c690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0500b4ae0 tx=0x7fa04002b880 comp rx=0 tx=0).stop 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 -- 172.21.15.181:0/708962912 shutdown_connections 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa05015eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 --2- 172.21.15.181:0/708962912 >> v2:172.21.15.187:3300/0 conn(0x7fa0500b4050 0x7fa0500cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 --2- 172.21.15.181:0/708962912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa05006c690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.882+0000 7fa058559640 1 -- 172.21.15.181:0/708962912 >> 172.21.15.181:0/708962912 conn(0x7fa0500672c0 msgr2=0x7fa0500676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.883+0000 7fa058559640 1 -- 172.21.15.181:0/708962912 shutdown_connections 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.883+0000 7fa058559640 1 -- 172.21.15.181:0/708962912 wait complete. 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 Processor -- start 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 -- start start 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 0x7fa05014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa0500c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0500cfb20 con 0x7fa050155990 2024-03-29T04:09:27.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0500cf9a0 con 0x7fa0500b4050 2024-03-29T04:09:27.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.884+0000 7fa058559640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0500b4500 con 0x7fa0500d0300 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34138/0 (socket says 172.21.15.181:34138) 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 -- 172.21.15.181:0/1601252055 learned_addr learned my addr 172.21.15.181:0/1601252055 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa0562f5640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 0x7fa05014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa055af4640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa0500c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 -- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 msgr2=0x7fa0500c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa0500c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 -- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 msgr2=0x7fa05014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 0x7fa05014b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 -- 172.21.15.181:0/1601252055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0500c6770 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa0562f5640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 0x7fa05014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa056af6640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa04800efc0 tx=0x7fa04800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048018030 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04800cdf0 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048003bb0 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0500c6a50 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.885+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0500c7010 con 0x7fa050155990 2024-03-29T04:09:27.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.886+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa050075a10 con 0x7fa050155990 2024-03-29T04:09:27.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.887+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa04801b070 con 0x7fa050155990 2024-03-29T04:09:27.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.887+0000 7fa046ffd640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 0x7fa03406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:27.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.887+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa04808c1e0 con 0x7fa050155990 2024-03-29T04:09:27.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.887+0000 7fa0562f5640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 0x7fa03406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:27.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.887+0000 7fa0562f5640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 0x7fa03406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa040036000 tx=0x7fa04003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:27.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:27.891+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa048010040 con 0x7fa050155990 2024-03-29T04:09:28.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.066+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0500cc580 con 0x7fa050155990 2024-03-29T04:09:28.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.067+0000 7fa046ffd640 1 -- 172.21.15.181:0/1601252055 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fa048059f00 con 0x7fa050155990 2024-03-29T04:09:28.047 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:28.047 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 msgr2=0x7fa03406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 0x7fa03406e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa040036000 tx=0x7fa04003d040 comp rx=0 tx=0).stop 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 msgr2=0x7fa0500c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa04800efc0 tx=0x7fa04800c490 comp rx=0 tx=0).stop 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 shutdown_connections 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa050155990 0x7fa0500c7e80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa03406c530 0x7fa03406e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0500d0300 0x7fa0500c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 --2- 172.21.15.181:0/1601252055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0500b4050 0x7fa05014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 >> 172.21.15.181:0/1601252055 conn(0x7fa0500672c0 msgr2=0x7fa0500b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 shutdown_connections 2024-03-29T04:09:28.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.069+0000 7fa058559640 1 -- 172.21.15.181:0/1601252055 wait complete. 2024-03-29T04:09:28.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:28.061 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.185+0000 7fdf696c5640 1 Processor -- start 2024-03-29T04:09:28.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.185+0000 7fdf696c5640 1 -- start start 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf696c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf640ba090 0x7fdf640d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf696c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 0x7fdf640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf696c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf640d2ff0 con 0x7fdf640b5dc0 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf696c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fdf640d32d0 con 0x7fdf640ba090 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf696c5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdf640d35b0 con 0x7fdf640b73d0 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf640ba090 0x7fdf640d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf640ba090 0x7fdf640d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34148/0 (socket says 172.21.15.181:34148) 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf62ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdf640b5dc0 0x7fdf640b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56962/0 (socket says 172.21.15.181:56962) 2024-03-29T04:09:28.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 -- 172.21.15.181:0/4215628211 learned_addr learned my addr 172.21.15.181:0/4215628211 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf637fe640 1 --2- 172.21.15.181:0/4215628211 >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 0x7fdf640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 -- 172.21.15.181:0/4215628211 >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 msgr2=0x7fdf640b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 --2- 172.21.15.181:0/4215628211 >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 0x7fdf640b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 -- 172.21.15.181:0/4215628211 >> v1:172.21.15.181:6789/0 conn(0x7fdf640b5dc0 legacy=0x7fdf640b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf627fc640 1 -- 172.21.15.181:0/4215628211 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf640d3890 con 0x7fdf640ba090 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf61ffb640 1 -- 172.21.15.181:0/4215628211 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162021771 0 0) 0x7fdf640d2ff0 con 0x7fdf640b5dc0 2024-03-29T04:09:28.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.186+0000 7fdf637fe640 1 --2- 172.21.15.181:0/4215628211 >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 0x7fdf640b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.187+0000 7fdf627fc640 1 --2- 172.21.15.181:0/4215628211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640ba090 0x7fdf640d2680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009870 tx=0x7fdf5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba8d4db63bb3e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.187+0000 7fdf61ffb640 1 -- 172.21.15.181:0/4215628211 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58034070 con 0x7fdf640ba090 2024-03-29T04:09:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.187+0000 7fdf61ffb640 1 -- 172.21.15.181:0/4215628211 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf58030860 con 0x7fdf640ba090 2024-03-29T04:09:28.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.187+0000 7fdf61ffb640 1 -- 172.21.15.181:0/4215628211 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58030c30 con 0x7fdf640ba090 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.187+0000 7fdf696c5640 1 -- 172.21.15.181:0/4215628211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640ba090 msgr2=0x7fdf640d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 --2- 172.21.15.181:0/4215628211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640ba090 0x7fdf640d2680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009870 tx=0x7fdf5802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 -- 172.21.15.181:0/4215628211 shutdown_connections 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 --2- 172.21.15.181:0/4215628211 >> v2:172.21.15.187:3300/0 conn(0x7fdf640b73d0 0x7fdf640b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 --2- 172.21.15.181:0/4215628211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640ba090 0x7fdf640d2680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 -- 172.21.15.181:0/4215628211 >> 172.21.15.181:0/4215628211 conn(0x7fdf640678f0 msgr2=0x7fdf640d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 -- 172.21.15.181:0/4215628211 shutdown_connections 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.188+0000 7fdf696c5640 1 -- 172.21.15.181:0/4215628211 wait complete. 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.189+0000 7fdf696c5640 1 Processor -- start 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.189+0000 7fdf696c5640 1 -- start start 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf640b5dc0 0x7fdf64155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf640ba090 0x7fdf6415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6415e880 con 0x7fdf640b73d0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf640d4090 con 0x7fdf640ba090 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6415ea00 con 0x7fdf640b5dc0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34158/0 (socket says 172.21.15.181:34158) 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 -- 172.21.15.181:0/773761020 learned_addr learned my addr 172.21.15.181:0/773761020 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf62ffd640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf640b5dc0 0x7fdf64155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf637fe640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf640ba090 0x7fdf6415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 -- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf640b5dc0 msgr2=0x7fdf64155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf640b5dc0 0x7fdf64155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 -- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf640ba090 msgr2=0x7fdf6415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf640ba090 0x7fdf6415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 -- 172.21.15.181:0/773761020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf640cb4b0 con 0x7fdf640b73d0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf627fc640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58034040 con 0x7fdf640b73d0 2024-03-29T04:09:28.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf58039d90 con 0x7fdf640b73d0 2024-03-29T04:09:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf640cb790 con 0x7fdf640b73d0 2024-03-29T04:09:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58043630 con 0x7fdf640b73d0 2024-03-29T04:09:28.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.190+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf64160010 con 0x7fdf640b73d0 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.191+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf64075890 con 0x7fdf640b73d0 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.192+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdf5804a060 con 0x7fdf640b73d0 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.192+0000 7fdf4b7fe640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 0x7fdf5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.192+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdf580ad8d0 con 0x7fdf640b73d0 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.192+0000 7fdf62ffd640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 0x7fdf5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.192+0000 7fdf62ffd640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 0x7fdf5006e9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000f60 tx=0x7fdf54007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.196+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdf5803e070 con 0x7fdf640b73d0 2024-03-29T04:09:28.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.374+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf6406b580 con 0x7fdf640b73d0 2024-03-29T04:09:28.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.374+0000 7fdf4b7fe640 1 -- 172.21.15.181:0/773761020 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fdf5807bbb0 con 0x7fdf640b73d0 2024-03-29T04:09:28.354 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 msgr2=0x7fdf5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 0x7fdf5006e9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000f60 tx=0x7fdf54007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 msgr2=0x7fdf641574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf580046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 shutdown_connections 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf640ba090 0x7fdf6415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf640b73d0 0x7fdf641574a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdf5006c530 0x7fdf5006e9f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 --2- 172.21.15.181:0/773761020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf640b5dc0 0x7fdf64155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 >> 172.21.15.181:0/773761020 conn(0x7fdf640678f0 msgr2=0x7fdf640b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 shutdown_connections 2024-03-29T04:09:28.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.376+0000 7fdf696c5640 1 -- 172.21.15.181:0/773761020 wait complete. 2024-03-29T04:09:28.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.491+0000 7f488d4c8640 1 Processor -- start 2024-03-29T04:09:28.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 -- start start 2024-03-29T04:09:28.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 0x7f48880b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 0x7f48880d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48880b66e0 con 0x7f488806c620 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f48880d2ff0 con 0x7f48880ba090 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f488d4c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48880d32d0 con 0x7f48880b5dc0 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f48877fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f488806c620 0x7f488806cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56976/0 (socket says 172.21.15.181:56976) 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.492+0000 7f48877fe640 1 -- 172.21.15.181:0/4118171801 learned_addr learned my addr 172.21.15.181:0/4118171801 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4886ffd640 1 --2- 172.21.15.181:0/4118171801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 0x7f48880b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f48867fc640 1 --2- 172.21.15.181:0/4118171801 >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 0x7f48880d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4885ffb640 1 -- 172.21.15.181:0/4118171801 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2702786046 0 0) 0x7f48880b66e0 con 0x7f488806c620 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4885ffb640 1 -- 172.21.15.181:0/4118171801 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4870003520 con 0x7f488806c620 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4886ffd640 1 -- 172.21.15.181:0/4118171801 >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 msgr2=0x7f48880d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4886ffd640 1 --2- 172.21.15.181:0/4118171801 >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 0x7f48880d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4886ffd640 1 -- 172.21.15.181:0/4118171801 >> v1:172.21.15.181:6789/0 conn(0x7f488806c620 legacy=0x7f488806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f4886ffd640 1 -- 172.21.15.181:0/4118171801 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48880d35b0 con 0x7f48880b5dc0 2024-03-29T04:09:28.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.493+0000 7f48867fc640 1 --2- 172.21.15.181:0/4118171801 >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 0x7f48880d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f4886ffd640 1 --2- 172.21.15.181:0/4118171801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 0x7f48880b61a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4878009d90 tx=0x7f487802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5474327fdfa3c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f4885ffb640 1 -- 172.21.15.181:0/4118171801 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878039070 con 0x7f48880b5dc0 2024-03-29T04:09:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f4885ffb640 1 -- 172.21.15.181:0/4118171801 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487802cc50 con 0x7f48880b5dc0 2024-03-29T04:09:28.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f4885ffb640 1 -- 172.21.15.181:0/4118171801 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878034440 con 0x7f48880b5dc0 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 -- 172.21.15.181:0/4118171801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 msgr2=0x7f48880b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 --2- 172.21.15.181:0/4118171801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 0x7f48880b61a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4878009d90 tx=0x7f487802bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 -- 172.21.15.181:0/4118171801 shutdown_connections 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 --2- 172.21.15.181:0/4118171801 >> v2:172.21.15.187:3300/0 conn(0x7f48880ba090 0x7f48880d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 --2- 172.21.15.181:0/4118171801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880b5dc0 0x7f48880b61a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 -- 172.21.15.181:0/4118171801 >> 172.21.15.181:0/4118171801 conn(0x7f48880678f0 msgr2=0x7f48880d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 -- 172.21.15.181:0/4118171801 shutdown_connections 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.494+0000 7f488d4c8640 1 -- 172.21.15.181:0/4118171801 wait complete. 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 Processor -- start 2024-03-29T04:09:28.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 -- start start 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48880b5dc0 0x7f4888160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880ba090 0x7f4888160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48880d4090 con 0x7f488806c620 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48880d3b70 con 0x7f48880b5dc0 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f488d4c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48880b83e0 con 0x7f48880ba090 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f4886ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f4886ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34174/0 (socket says 172.21.15.181:34174) 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f4886ffd640 1 -- 172.21.15.181:0/2705545944 learned_addr learned my addr 172.21.15.181:0/2705545944 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f48877fe640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880ba090 0x7f4888160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.496+0000 7f48867fc640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48880b5dc0 0x7f4888160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 -- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880ba090 msgr2=0x7f4888160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880ba090 0x7f4888160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 -- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48880b5dc0 msgr2=0x7f4888160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48880b5dc0 0x7f4888160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 -- 172.21.15.181:0/2705545944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48880cc070 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f4886ffd640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4878037040 tx=0x7f4878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878044020 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487802ca50 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48880cc350 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878039040 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.497+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48880cc910 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.498+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888075890 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.498+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4878047020 con 0x7f488806c620 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.499+0000 7f486f7fe640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 0x7f487406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.499+0000 7f48867fc640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 0x7f487406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.499+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48780ae050 con 0x7f488806c620 2024-03-29T04:09:28.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.499+0000 7f48867fc640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 0x7f487406e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f487c009560 tx=0x7f487c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.503+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f487804e030 con 0x7f488806c620 2024-03-29T04:09:28.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.679+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48880b38f0 con 0x7f488806c620 2024-03-29T04:09:28.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.679+0000 7f486f7fe640 1 -- 172.21.15.181:0/2705545944 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f487807bd70 con 0x7f488806c620 2024-03-29T04:09:28.659 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 msgr2=0x7f487406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 0x7f487406e9f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f487c009560 tx=0x7f487c009190 comp rx=0 tx=0).stop 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 msgr2=0x7f48880cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4878037040 tx=0x7f4878004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 shutdown_connections 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f487406c530 0x7f487406e9f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48880ba090 0x7f4888160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48880b5dc0 0x7f4888160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 --2- 172.21.15.181:0/2705545944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488806c620 0x7f48880cd7b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 >> 172.21.15.181:0/2705545944 conn(0x7f48880678f0 msgr2=0x7f48880cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 shutdown_connections 2024-03-29T04:09:28.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.681+0000 7f488d4c8640 1 -- 172.21.15.181:0/2705545944 wait complete. 2024-03-29T04:09:28.672 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:28.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.796+0000 7fc6f7022640 1 Processor -- start 2024-03-29T04:09:28.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.796+0000 7fc6f7022640 1 -- start start 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f7022640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f0144cb0 0x7fc6f00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f7022640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc6f015f800 0x7fc6f0072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f7022640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6f00b35e0 con 0x7fc6f0072b80 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f7022640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc6f00b3e80 con 0x7fc6f015f800 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f7022640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f00b38c0 con 0x7fc6f0144cb0 2024-03-29T04:09:28.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc6f015f800 0x7fc6f0072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc6f015f800 0x7fc6f0072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34184/0 (socket says 172.21.15.181:34184) 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 -- 172.21.15.181:0/1072698154 learned_addr learned my addr 172.21.15.181:0/1072698154 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6f4dbe640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f0144cb0 0x7fc6f00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 -- 172.21.15.181:0/1072698154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f0144cb0 msgr2=0x7fc6f00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f0144cb0 0x7fc6f00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 -- 172.21.15.181:0/1072698154 >> v1:172.21.15.187:6789/0 conn(0x7fc6f0072b80 legacy=0x7fc6f0071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.797+0000 7fc6effff640 1 -- 172.21.15.181:0/1072698154 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f00b3ba0 con 0x7fc6f015f800 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6effff640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f015f800 0x7fc6f0072640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009870 tx=0x7fc6dc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4093f9b263b6941 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6ef7fe640 1 -- 172.21.15.181:0/1072698154 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc034070 con 0x7fc6f015f800 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6ef7fe640 1 -- 172.21.15.181:0/1072698154 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc030840 con 0x7fc6f015f800 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6ef7fe640 1 -- 172.21.15.181:0/1072698154 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc030c10 con 0x7fc6f015f800 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6f7022640 1 -- 172.21.15.181:0/1072698154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f015f800 msgr2=0x7fc6f0072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6f7022640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f015f800 0x7fc6f0072640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009870 tx=0x7fc6dc02b980 comp rx=0 tx=0).stop 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6f7022640 1 -- 172.21.15.181:0/1072698154 shutdown_connections 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.798+0000 7fc6f7022640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f015f800 0x7fc6f0072640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.799+0000 7fc6f7022640 1 --2- 172.21.15.181:0/1072698154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f0144cb0 0x7fc6f00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.799+0000 7fc6f7022640 1 -- 172.21.15.181:0/1072698154 >> 172.21.15.181:0/1072698154 conn(0x7fc6f00b5b00 msgr2=0x7fc6f006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.799+0000 7fc6f7022640 1 -- 172.21.15.181:0/1072698154 shutdown_connections 2024-03-29T04:09:28.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.799+0000 7fc6f7022640 1 -- 172.21.15.181:0/1072698154 wait complete. 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.800+0000 7fc6f7022640 1 Processor -- start 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.800+0000 7fc6f7022640 1 -- start start 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.800+0000 7fc6f7022640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f0144cb0 0x7fc6f015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f015f800 0x7fc6f015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f00cf060 con 0x7fc6f0072b80 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f015fc40 con 0x7fc6f0144cb0 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f00cf1e0 con 0x7fc6f015f800 2024-03-29T04:09:28.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34198/0 (socket says 172.21.15.181:34198) 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 -- 172.21.15.181:0/4208056122 learned_addr learned my addr 172.21.15.181:0/4208056122 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6effff640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f0144cb0 0x7fc6f015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f55bf640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f015f800 0x7fc6f015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 -- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f015f800 msgr2=0x7fc6f015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f015f800 0x7fc6f015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 -- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f0144cb0 msgr2=0x7fc6f015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f0144cb0 0x7fc6f015c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 -- 172.21.15.181:0/4208056122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f00af1d0 con 0x7fc6f0072b80 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f4dbe640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400e9e0 tx=0x7fc6e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4017070 con 0x7fc6f0072b80 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e400c970 con 0x7fc6f0072b80 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400cd60 con 0x7fc6f0072b80 2024-03-29T04:09:28.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f00ae650 con 0x7fc6f0072b80 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.801+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f00aec10 con 0x7fc6f0072b80 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.802+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f00af920 con 0x7fc6f0072b80 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.803+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc6e401a070 con 0x7fc6f0072b80 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.803+0000 7fc6ed7fa640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 0x7fc6d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.803+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc6e408b890 con 0x7fc6f0072b80 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.803+0000 7fc6effff640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 0x7fc6d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:28.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.804+0000 7fc6effff640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 0x7fc6d406e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc036000 tx=0x7fc6dc03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:28.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.808+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc6f00af920 con 0x7fc6f0072b80 2024-03-29T04:09:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.986+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6f0075890 con 0x7fc6f0072b80 2024-03-29T04:09:28.966 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.986+0000 7fc6ed7fa640 1 -- 172.21.15.181:0/4208056122 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc6e40595b0 con 0x7fc6f0072b80 2024-03-29T04:09:28.966 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 msgr2=0x7fc6d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 0x7fc6d406e9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc036000 tx=0x7fc6dc03b040 comp rx=0 tx=0).stop 2024-03-29T04:09:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 msgr2=0x7fc6f0153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400e9e0 tx=0x7fc6e400eea0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 shutdown_connections 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6d406c530 0x7fc6d406e9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6f015f800 0x7fc6f015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f0144cb0 0x7fc6f015c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 --2- 172.21.15.181:0/4208056122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6f0072b80 0x7fc6f0153a60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.988+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 >> 172.21.15.181:0/4208056122 conn(0x7fc6f00b5b00 msgr2=0x7fc6f0071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.989+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 shutdown_connections 2024-03-29T04:09:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:28.989+0000 7fc6f7022640 1 -- 172.21.15.181:0/4208056122 wait complete. 2024-03-29T04:09:28.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.102+0000 7fe3d9a74640 1 Processor -- start 2024-03-29T04:09:29.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.102+0000 7fe3d9a74640 1 -- start start 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d9a74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3d40d27a0 0x7fe3d40d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d9a74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d48a0 0x7fe3d415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d9a74640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d40b3c30 con 0x7fe3d415c000 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d9a74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d40d2c00 con 0x7fe3d40d27a0 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d9a74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40d4cf0 con 0x7fe3d40d48a0 2024-03-29T04:09:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3d40d27a0 0x7fe3d40d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3d40d27a0 0x7fe3d40d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34202/0 (socket says 172.21.15.181:34202) 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/927063514 learned_addr learned my addr 172.21.15.181:0/927063514 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d3fff640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d48a0 0x7fe3d415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/927063514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d48a0 msgr2=0x7fe3d415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d48a0 0x7fe3d415edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/927063514 >> v1:172.21.15.187:6789/0 conn(0x7fe3d415c000 legacy=0x7fe3d40b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d27fc640 1 -- 172.21.15.181:0/927063514 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389112613 0 0) 0x7fe3d40b3c30 con 0x7fe3d415c000 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.103+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/927063514 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d4162110 con 0x7fe3d40d27a0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d2ffd640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d27a0 0x7fe3d40d4360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009870 tx=0x7fe3c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aedfacd7022fb5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d27fc640 1 -- 172.21.15.181:0/927063514 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4035020 con 0x7fe3d40d27a0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d27fc640 1 -- 172.21.15.181:0/927063514 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4031640 con 0x7fe3d40d27a0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d27fc640 1 -- 172.21.15.181:0/927063514 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c402d040 con 0x7fe3d40d27a0 2024-03-29T04:09:29.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 -- 172.21.15.181:0/927063514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d27a0 msgr2=0x7fe3d40d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d27a0 0x7fe3d40d4360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009870 tx=0x7fe3c402b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 -- 172.21.15.181:0/927063514 shutdown_connections 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d48a0 0x7fe3d415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/927063514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d27a0 0x7fe3d40d4360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.104+0000 7fe3d9a74640 1 -- 172.21.15.181:0/927063514 >> 172.21.15.181:0/927063514 conn(0x7fe3d40676e0 msgr2=0x7fe3d4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.105+0000 7fe3d9a74640 1 -- 172.21.15.181:0/927063514 shutdown_connections 2024-03-29T04:09:29.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.105+0000 7fe3d9a74640 1 -- 172.21.15.181:0/927063514 wait complete. 2024-03-29T04:09:29.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 Processor -- start 2024-03-29T04:09:29.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 -- start start 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d27a0 0x7fe3d40b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3d415c000 0x7fe3d40c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4162990 con 0x7fe3d40d48a0 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d41626b0 con 0x7fe3d415c000 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.106+0000 7fe3d9a74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d4162c70 con 0x7fe3d40d27a0 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34216/0 (socket says 172.21.15.181:34216) 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/887346872 learned_addr learned my addr 172.21.15.181:0/887346872 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d3fff640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3d415c000 0x7fe3d40c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d37fe640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d27a0 0x7fe3d40b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d27a0 msgr2=0x7fe3d40b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d27a0 0x7fe3d40b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3d415c000 msgr2=0x7fe3d40c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3d415c000 0x7fe3d40c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 -- 172.21.15.181:0/887346872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40b5a80 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d2ffd640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4040070 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c40399f0 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4035040 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d40b5d00 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.107+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d40b62c0 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.108+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d4067ac0 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.109+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe3c404b060 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.109+0000 7fe3bbfff640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 0x7fe3c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.109+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe3c40ad110 con 0x7fe3d40d48a0 2024-03-29T04:09:29.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.109+0000 7fe3d37fe640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 0x7fe3c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.109+0000 7fe3d37fe640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 0x7fe3c006e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8000f60 tx=0x7fe3c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.113+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe3d4067ac0 con 0x7fe3d40d48a0 2024-03-29T04:09:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.290+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe3d4075890 con 0x7fe3d40d48a0 2024-03-29T04:09:29.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.290+0000 7fe3bbfff640 1 -- 172.21.15.181:0/887346872 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fe3d40b5a80 con 0x7fe3d40d48a0 2024-03-29T04:09:29.270 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:29.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 msgr2=0x7fe3c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 0x7fe3c006e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8000f60 tx=0x7fe3c8009040 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 msgr2=0x7fe3d40c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4000c00 tx=0x7fe3c40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 shutdown_connections 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3d415c000 0x7fe3d40c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d40d48a0 0x7fe3d40c7ea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe3c006c530 0x7fe3c006e9f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 --2- 172.21.15.181:0/887346872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3d40d27a0 0x7fe3d40b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.292+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 >> 172.21.15.181:0/887346872 conn(0x7fe3d40676e0 msgr2=0x7fe3d415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.293+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 shutdown_connections 2024-03-29T04:09:29.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.293+0000 7fe3d9a74640 1 -- 172.21.15.181:0/887346872 wait complete. 2024-03-29T04:09:29.283 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:09:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.406+0000 7f7f7b18d640 1 Processor -- start 2024-03-29T04:09:29.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.406+0000 7f7f7b18d640 1 -- start start 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7b18d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f7415ec90 0x7f7f7415f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7b18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f74143e50 0x7f7f741442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7b18d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f74162200 con 0x7f7f740ceb90 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7b18d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f7f74162b10 con 0x7f7f7415ec90 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7b18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7415fa20 con 0x7f7f74143e50 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f73fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f7415ec90 0x7f7f7415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f78f29640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7f740ceb90 0x7f7f740cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56984/0 (socket says 172.21.15.181:56984) 2024-03-29T04:09:29.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f78f29640 1 -- 172.21.15.181:0/275162407 learned_addr learned my addr 172.21.15.181:0/275162407 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f7972a640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f74143e50 0x7f7f741442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f73fff640 1 -- 172.21.15.181:0/275162407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f74143e50 msgr2=0x7f7f741442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f73fff640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f74143e50 0x7f7f741442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f73fff640 1 -- 172.21.15.181:0/275162407 >> v1:172.21.15.181:6789/0 conn(0x7f7f740ceb90 legacy=0x7f7f740cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f737fe640 1 -- 172.21.15.181:0/275162407 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235131138 0 0) 0x7f7f74162200 con 0x7f7f740ceb90 2024-03-29T04:09:29.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.407+0000 7f7f73fff640 1 -- 172.21.15.181:0/275162407 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f741624c0 con 0x7f7f7415ec90 2024-03-29T04:09:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.408+0000 7f7f73fff640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7415ec90 0x7f7f7415f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8853d8f85045f711 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.408+0000 7f7f737fe640 1 -- 172.21.15.181:0/275162407 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035020 con 0x7f7f7415ec90 2024-03-29T04:09:29.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.408+0000 7f7f737fe640 1 -- 172.21.15.181:0/275162407 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60031830 con 0x7f7f7415ec90 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.408+0000 7f7f737fe640 1 -- 172.21.15.181:0/275162407 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60031c00 con 0x7f7f7415ec90 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 -- 172.21.15.181:0/275162407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7415ec90 msgr2=0x7f7f7415f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7415ec90 0x7f7f7415f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f6002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 -- 172.21.15.181:0/275162407 shutdown_connections 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f74143e50 0x7f7f741442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/275162407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f7415ec90 0x7f7f7415f110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 -- 172.21.15.181:0/275162407 >> 172.21.15.181:0/275162407 conn(0x7f7f740b3cb0 msgr2=0x7f7f740b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 -- 172.21.15.181:0/275162407 shutdown_connections 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.409+0000 7f7f7b18d640 1 -- 172.21.15.181:0/275162407 wait complete. 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.410+0000 7f7f7b18d640 1 Processor -- start 2024-03-29T04:09:29.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.410+0000 7f7f7b18d640 1 -- start start 2024-03-29T04:09:29.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f740ceb90 0x7f7f74157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f7415ec90 0x7f7f740cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74158f10 con 0x7f7f74143e50 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74162fa0 con 0x7f7f7415ec90 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f74159090 con 0x7f7f740ceb90 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34228/0 (socket says 172.21.15.181:34228) 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 -- 172.21.15.181:0/191986254 learned_addr learned my addr 172.21.15.181:0/191986254 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7972a640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f7415ec90 0x7f7f740cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f78f29640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f740ceb90 0x7f7f74157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 -- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f740ceb90 msgr2=0x7f7f74157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f740ceb90 0x7f7f74157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 -- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f7415ec90 msgr2=0x7f7f740cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f7415ec90 0x7f7f740cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 -- 172.21.15.181:0/191986254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f74158340 con 0x7f7f74143e50 2024-03-29T04:09:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f73fff640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035040 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60004230 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6003a640 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f740c8c40 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.411+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f740c9c80 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.412+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74075900 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.413+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f60038020 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.413+0000 7f7f717fa640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 0x7f7f5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.413+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7f600ad5d0 con 0x7f7f74143e50 2024-03-29T04:09:29.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.413+0000 7f7f78f29640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 0x7f7f5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.414+0000 7f7f78f29640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 0x7f7f5806e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000f60 tx=0x7f7f68007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.417+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f74158340 con 0x7f7f74143e50 2024-03-29T04:09:29.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.604+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7f74162200 con 0x7f7f74143e50 2024-03-29T04:09:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.604+0000 7f7f717fa640 1 -- 172.21.15.181:0/191986254 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f7f6007b260 con 0x7f7f74143e50 2024-03-29T04:09:29.584 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 msgr2=0x7f7f5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 0x7f7f5806e9f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7f68000f60 tx=0x7f7f68007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 msgr2=0x7f7f740ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f600046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 shutdown_connections 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f7415ec90 0x7f7f740cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74143e50 0x7f7f740ca760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7f5806c530 0x7f7f5806e9f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 --2- 172.21.15.181:0/191986254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f740ceb90 0x7f7f74157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 >> 172.21.15.181:0/191986254 conn(0x7f7f740b3cb0 msgr2=0x7f7f740ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 shutdown_connections 2024-03-29T04:09:29.587 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.606+0000 7f7f7b18d640 1 -- 172.21.15.181:0/191986254 wait complete. 2024-03-29T04:09:29.596 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-03-29T04:09:29.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:29.597 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-29T04:09:29.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi181.stderr:ceph-fuse[44712]: fuse finished with error 0 and tester_r 0 2024-03-29T04:09:29.627 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:09:29.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:09:29.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:29.726 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-29T04:09:29.749 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:29.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.910+0000 7f2c204a5640 1 Processor -- start 2024-03-29T04:09:29.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.910+0000 7f2c204a5640 1 -- start start 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c204a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c180b3f20 0x7f2c1815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c204a5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2c18141da0 0x7f2c180d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c204a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1815fbe0 con 0x7f2c180b4300 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c204a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f2c180d1cd0 con 0x7f2c180b3f20 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c204a5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2c181625f0 con 0x7f2c18141da0 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1e241640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c180b3f20 0x7f2c1815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1ea42640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2c180b4300 0x7f2c180d2640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56988/0 (socket says 172.21.15.181:56988) 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1ea42640 1 -- 172.21.15.181:0/534901217 learned_addr learned my addr 172.21.15.181:0/534901217 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1da40640 1 --2- 172.21.15.181:0/534901217 >> v2:172.21.15.187:3300/0 conn(0x7f2c18141da0 0x7f2c180d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1e241640 1 -- 172.21.15.181:0/534901217 >> v2:172.21.15.187:3300/0 conn(0x7f2c18141da0 msgr2=0x7f2c180d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1e241640 1 --2- 172.21.15.181:0/534901217 >> v2:172.21.15.187:3300/0 conn(0x7f2c18141da0 0x7f2c180d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1e241640 1 -- 172.21.15.181:0/534901217 >> v1:172.21.15.181:6789/0 conn(0x7f2c180b4300 legacy=0x7f2c180d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1e241640 1 -- 172.21.15.181:0/534901217 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181628d0 con 0x7f2c180b3f20 2024-03-29T04:09:29.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.911+0000 7f2c1d23f640 1 -- 172.21.15.181:0/534901217 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3507150436 0 0) 0x7f2c10001110 con 0x7f2c180b4300 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.912+0000 7f2c1e241640 1 --2- 172.21.15.181:0/534901217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b3f20 0x7f2c1815f6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c180cfd70 tx=0x7f2c0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf2a39eb3e075ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.912+0000 7f2c1d23f640 1 -- 172.21.15.181:0/534901217 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08039070 con 0x7f2c180b3f20 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.912+0000 7f2c1d23f640 1 -- 172.21.15.181:0/534901217 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0802cc80 con 0x7f2c180b3f20 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.912+0000 7f2c1d23f640 1 -- 172.21.15.181:0/534901217 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08034660 con 0x7f2c180b3f20 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 -- 172.21.15.181:0/534901217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b3f20 msgr2=0x7f2c1815f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 --2- 172.21.15.181:0/534901217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b3f20 0x7f2c1815f6a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c180cfd70 tx=0x7f2c0802bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 -- 172.21.15.181:0/534901217 shutdown_connections 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 --2- 172.21.15.181:0/534901217 >> v2:172.21.15.187:3300/0 conn(0x7f2c18141da0 0x7f2c180d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 --2- 172.21.15.181:0/534901217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b3f20 0x7f2c1815f6a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 -- 172.21.15.181:0/534901217 >> 172.21.15.181:0/534901217 conn(0x7f2c180672c0 msgr2=0x7f2c180676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 -- 172.21.15.181:0/534901217 shutdown_connections 2024-03-29T04:09:29.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.913+0000 7f2c204a5640 1 -- 172.21.15.181:0/534901217 wait complete. 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.914+0000 7f2c204a5640 1 Processor -- start 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.914+0000 7f2c204a5640 1 -- start start 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c180b3f20 0x7f2c18159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c18141da0 0x7f2c1806fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180d0060 con 0x7f2c180b4300 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18162e90 con 0x7f2c180b3f20 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c204a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2c18162210 con 0x7f2c18141da0 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34258/0 (socket says 172.21.15.181:34258) 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1e241640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c180b3f20 0x7f2c18159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 -- 172.21.15.181:0/2463130295 learned_addr learned my addr 172.21.15.181:0/2463130295 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1ea42640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c18141da0 0x7f2c1806fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 -- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c18141da0 msgr2=0x7f2c1806fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c18141da0 0x7f2c1806fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 -- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c180b3f20 msgr2=0x7f2c18159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c180b3f20 0x7f2c18159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 -- 172.21.15.181:0/2463130295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1806e270 con 0x7f2c180b4300 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.915+0000 7f2c1da40640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00e840 tx=0x7f2c0c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.916+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c017070 con 0x7f2c180b4300 2024-03-29T04:09:29.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.916+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c00c940 con 0x7f2c180b4300 2024-03-29T04:09:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.916+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1806d620 con 0x7f2c180b4300 2024-03-29T04:09:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.916+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00cd10 con 0x7f2c180b4300 2024-03-29T04:09:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.916+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1806e550 con 0x7f2c180b4300 2024-03-29T04:09:29.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.917+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18075a10 con 0x7f2c180b4300 2024-03-29T04:09:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.917+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2c0c01a070 con 0x7f2c180b4300 2024-03-29T04:09:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.917+0000 7f2c16ffd640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 0x7f2c0406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.918+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c0c08bbe0 con 0x7f2c180b4300 2024-03-29T04:09:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.918+0000 7f2c1e241640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 0x7f2c0406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:29.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.918+0000 7f2c1e241640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 0x7f2c0406e9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c08036000 tx=0x7f2c08030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:29.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:29.922+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2c0c0597d0 con 0x7f2c180b4300 2024-03-29T04:09:30.077 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.097+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c180cf5a0 con 0x7f2c180b4300 2024-03-29T04:09:30.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.098+0000 7f2c16ffd640 1 -- 172.21.15.181:0/2463130295 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f2c0c05dad0 con 0x7f2c180b4300 2024-03-29T04:09:30.078 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:30.079 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 msgr2=0x7f2c0406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 0x7f2c0406e9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c08036000 tx=0x7f2c08030000 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 msgr2=0x7f2c1806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00e840 tx=0x7f2c0c00ed00 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 shutdown_connections 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c0406c530 0x7f2c0406e9f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c18141da0 0x7f2c1806fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c180b4300 0x7f2c1806f920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 --2- 172.21.15.181:0/2463130295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c180b3f20 0x7f2c18159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 >> 172.21.15.181:0/2463130295 conn(0x7f2c180672c0 msgr2=0x7f2c180ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 shutdown_connections 2024-03-29T04:09:30.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.100+0000 7f2c204a5640 1 -- 172.21.15.181:0/2463130295 wait complete. 2024-03-29T04:09:30.090 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:30.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.213+0000 7ff6474d3640 1 Processor -- start 2024-03-29T04:09:30.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.213+0000 7ff6474d3640 1 -- start start 2024-03-29T04:09:30.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff6474d3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff640147410 0x7ff640071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff6474d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 0x7ff6400ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff6474d3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64015e9a0 con 0x7ff640145220 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff6474d3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ff64006e440 con 0x7ff640147410 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff6474d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff64006c5b0 con 0x7ff64006dfb0 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff64526f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff640147410 0x7ff640071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff644a6e640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff640145220 0x7ff64006d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:56990/0 (socket says 172.21.15.181:56990) 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff644a6e640 1 -- 172.21.15.181:0/833618144 learned_addr learned my addr 172.21.15.181:0/833618144 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff645a70640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 0x7ff6400ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff64526f640 1 -- 172.21.15.181:0/833618144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 msgr2=0x7ff6400ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff64526f640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 0x7ff6400ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff64526f640 1 -- 172.21.15.181:0/833618144 >> v1:172.21.15.181:6789/0 conn(0x7ff640145220 legacy=0x7ff64006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff64526f640 1 -- 172.21.15.181:0/833618144 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64006c800 con 0x7ff640147410 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff637fff640 1 -- 172.21.15.181:0/833618144 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388126931 0 0) 0x7ff64015e9a0 con 0x7ff640145220 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.214+0000 7ff645a70640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 0x7ff6400ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.215+0000 7ff64526f640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff640147410 0x7ff640071cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0097a0 tx=0x7ff63c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97110c776451278b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.215+0000 7ff637fff640 1 -- 172.21.15.181:0/833618144 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c035020 con 0x7ff640147410 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.215+0000 7ff637fff640 1 -- 172.21.15.181:0/833618144 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63c031830 con 0x7ff640147410 2024-03-29T04:09:30.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.215+0000 7ff637fff640 1 -- 172.21.15.181:0/833618144 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c031c00 con 0x7ff640147410 2024-03-29T04:09:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 -- 172.21.15.181:0/833618144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff640147410 msgr2=0x7ff640071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff640147410 0x7ff640071cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0097a0 tx=0x7ff63c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 -- 172.21.15.181:0/833618144 shutdown_connections 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff64006dfb0 0x7ff6400ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 --2- 172.21.15.181:0/833618144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff640147410 0x7ff640071cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 -- 172.21.15.181:0/833618144 >> 172.21.15.181:0/833618144 conn(0x7ff64006f820 msgr2=0x7ff64006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 -- 172.21.15.181:0/833618144 shutdown_connections 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.216+0000 7ff6474d3640 1 -- 172.21.15.181:0/833618144 wait complete. 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.217+0000 7ff6474d3640 1 Processor -- start 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.217+0000 7ff6474d3640 1 -- start start 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.217+0000 7ff6474d3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.217+0000 7ff6474d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff640145220 0x7ff6400b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff640147410 0x7ff6400b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff64006d0a0 con 0x7ff64006dfb0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff64006cdc0 con 0x7ff640145220 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff64006d380 con 0x7ff640147410 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34284/0 (socket says 172.21.15.181:34284) 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 -- 172.21.15.181:0/2185987262 learned_addr learned my addr 172.21.15.181:0/2185987262 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff644a6e640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff640145220 0x7ff6400b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff645a70640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff640147410 0x7ff6400b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 -- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff640147410 msgr2=0x7ff6400b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff640147410 0x7ff6400b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 -- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff640145220 msgr2=0x7ff6400b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff640145220 0x7ff6400b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 -- 172.21.15.181:0/2185987262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6401534c0 con 0x7ff64006dfb0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff64526f640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff63c034040 tx=0x7ff63c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c040070 con 0x7ff64006dfb0 2024-03-29T04:09:30.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63c03a850 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6400b74d0 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c035040 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.218+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6400ca2c0 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.219+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff640075a10 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.220+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff63c038020 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.220+0000 7ff635ffb640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 0x7ff62406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.220+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff63c0ad390 con 0x7ff64006dfb0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.220+0000 7ff644a6e640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 0x7ff62406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.221+0000 7ff644a6e640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 0x7ff62406e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6400b7d80 tx=0x7ff630007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.225+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff63c031800 con 0x7ff64006dfb0 2024-03-29T04:09:30.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.401+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff640071aa0 con 0x7ff64006dfb0 2024-03-29T04:09:30.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.402+0000 7ff635ffb640 1 -- 172.21.15.181:0/2185987262 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7ff63c07b670 con 0x7ff64006dfb0 2024-03-29T04:09:30.382 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:30.382 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 msgr2=0x7ff62406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 0x7ff62406e9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6400b7d80 tx=0x7ff630007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 msgr2=0x7ff640152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff63c034040 tx=0x7ff63c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 shutdown_connections 2024-03-29T04:09:30.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff62406c530 0x7ff62406e9f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff640147410 0x7ff6400b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff640145220 0x7ff6400b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 --2- 172.21.15.181:0/2185987262 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff64006dfb0 0x7ff640152b20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 >> 172.21.15.181:0/2185987262 conn(0x7ff64006f820 msgr2=0x7ff640145680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 shutdown_connections 2024-03-29T04:09:30.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.404+0000 7ff6474d3640 1 -- 172.21.15.181:0/2185987262 wait complete. 2024-03-29T04:09:30.395 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:09:30.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.518+0000 7f6616ad3640 1 Processor -- start 2024-03-29T04:09:30.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 -- start start 2024-03-29T04:09:30.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 0x7f66100d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 0x7f66100b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66100d2ff0 con 0x7f66100b5dc0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f66100d32d0 con 0x7f66100ba090 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6616ad3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66100d35b0 con 0x7f66100b73d0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f660ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 0x7f66100d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f661486f640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f66100b5dc0 0x7f66100b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:34958/0 (socket says 172.21.15.181:34958) 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f661486f640 1 -- 172.21.15.181:0/103495736 learned_addr learned my addr 172.21.15.181:0/103495736 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.519+0000 7f6615070640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 0x7f66100b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f6615070640 1 -- 172.21.15.181:0/103495736 >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 msgr2=0x7f66100d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f6615070640 1 --2- 172.21.15.181:0/103495736 >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 0x7f66100d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f6615070640 1 -- 172.21.15.181:0/103495736 >> v1:172.21.15.187:6789/0 conn(0x7f66100b5dc0 legacy=0x7f66100b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f6615070640 1 -- 172.21.15.181:0/103495736 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66100d3890 con 0x7f66100b73d0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f660f7fe640 1 -- 172.21.15.181:0/103495736 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3833056105 0 0) 0x7f66100d2ff0 con 0x7f66100b5dc0 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.520+0000 7f660ffff640 1 --2- 172.21.15.181:0/103495736 >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 0x7f66100d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:30.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6615070640 1 --2- 172.21.15.181:0/103495736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 0x7f66100b7860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6608009dc0 tx=0x7f660802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3529366574c063ab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f660f7fe640 1 -- 172.21.15.181:0/103495736 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608039070 con 0x7f66100b73d0 2024-03-29T04:09:30.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f660f7fe640 1 -- 172.21.15.181:0/103495736 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660802cc40 con 0x7f66100b73d0 2024-03-29T04:09:30.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f660f7fe640 1 -- 172.21.15.181:0/103495736 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608034440 con 0x7f66100b73d0 2024-03-29T04:09:30.505 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 -- 172.21.15.181:0/103495736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 msgr2=0x7f66100b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.505 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 --2- 172.21.15.181:0/103495736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 0x7f66100b7860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6608009dc0 tx=0x7f660802be90 comp rx=0 tx=0).stop 2024-03-29T04:09:30.505 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 -- 172.21.15.181:0/103495736 shutdown_connections 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 --2- 172.21.15.181:0/103495736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100b73d0 0x7f66100b7860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 --2- 172.21.15.181:0/103495736 >> v2:172.21.15.187:3300/0 conn(0x7f66100ba090 0x7f66100d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 -- 172.21.15.181:0/103495736 >> 172.21.15.181:0/103495736 conn(0x7f66100678f0 msgr2=0x7f66100d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 -- 172.21.15.181:0/103495736 shutdown_connections 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.521+0000 7f6616ad3640 1 -- 172.21.15.181:0/103495736 wait complete. 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.522+0000 7f6616ad3640 1 Processor -- start 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.522+0000 7f6616ad3640 1 -- start start 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66100b5dc0 0x7f66100ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 0x7f66101f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f661015e880 con 0x7f66100b73d0 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66100d4090 con 0x7f66100b5dc0 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6616ad3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f661015ea00 con 0x7f66100ba090 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f660ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 0x7f66101f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f661486f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66100b5dc0 0x7f66100ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6615070640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f660ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 0x7f66101f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34296/0 (socket says 172.21.15.181:34296) 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f6615070640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:50966/0 (socket says 172.21.15.181:50966) 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.523+0000 7f660ffff640 1 -- 172.21.15.181:0/2556550896 learned_addr learned my addr 172.21.15.181:0/2556550896 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 -- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66100b5dc0 msgr2=0x7f66100ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66100b5dc0 0x7f66100ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 -- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 msgr2=0x7f66101f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 0x7f66101f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 -- 172.21.15.181:0/2556550896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66101f4e60 con 0x7f66100ba090 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6615070640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6608033000 tx=0x7f6608004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608044020 con 0x7f66100ba090 2024-03-29T04:09:30.507 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660803a040 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6610161680 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608039050 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.524+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66100cbda0 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.525+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6610075890 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.526+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6608003c50 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.526+0000 7f660d7fa640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 0x7f65fc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.526+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f66080add30 con 0x7f66100ba090 2024-03-29T04:09:30.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.526+0000 7f661486f640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 0x7f65fc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:30.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.526+0000 7f661486f640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 0x7f65fc06e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6600000f60 tx=0x7f6600007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:30.514 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.535+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f660804e020 con 0x7f66100ba090 2024-03-29T04:09:30.729 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:30.750+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f661015de30 con 0x7f66100ba090 2024-03-29T04:09:31.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.355+0000 7f660d7fa640 1 -- 172.21.15.181:0/2556550896 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f66101f4e60 con 0x7f66100ba090 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 msgr2=0x7f65fc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 0x7f65fc06e9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6600000f60 tx=0x7f6600007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 msgr2=0x7f66101f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6608033000 tx=0x7f6608004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 shutdown_connections 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f65fc06c530 0x7f65fc06e9f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66100ba090 0x7f66101f5060 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66100b73d0 0x7f66101f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 --2- 172.21.15.181:0/2556550896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66100b5dc0 0x7f66100ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 >> 172.21.15.181:0/2556550896 conn(0x7f66100678f0 msgr2=0x7f66100b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 shutdown_connections 2024-03-29T04:09:31.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.358+0000 7f6616ad3640 1 -- 172.21.15.181:0/2556550896 wait complete. 2024-03-29T04:09:31.350 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-29T04:09:31.351 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:09:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.479+0000 7fc73e7db640 1 Processor -- start 2024-03-29T04:09:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.479+0000 7fc73e7db640 1 -- start start 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73e7db640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc7380718c0 0x7fc7380b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73e7db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc73815a710 0x7fc7380cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73e7db640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7380b4d70 con 0x7fc73815ab50 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73e7db640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc7380b41d0 con 0x7fc7380718c0 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73e7db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7380b35c0 con 0x7fc73815a710 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73cd78640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc73815ab50 0x7fc7380b4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57002/0 (socket says 172.21.15.181:57002) 2024-03-29T04:09:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc73cd78640 1 -- 172.21.15.181:0/3161093841 learned_addr learned my addr 172.21.15.181:0/3161093841 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc737fff640 1 --2- 172.21.15.181:0/3161093841 >> v2:172.21.15.187:3300/0 conn(0x7fc7380718c0 0x7fc7380b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc7377fe640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc73815a710 0x7fc7380cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc736ffd640 1 -- 172.21.15.181:0/3161093841 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114774359 0 0) 0x7fc7380b4d70 con 0x7fc73815ab50 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc736ffd640 1 -- 172.21.15.181:0/3161093841 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc720003520 con 0x7fc73815ab50 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc737fff640 1 -- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc73815a710 msgr2=0x7fc7380cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc737fff640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc73815a710 0x7fc7380cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc737fff640 1 -- 172.21.15.181:0/3161093841 >> v1:172.21.15.181:6789/0 conn(0x7fc73815ab50 legacy=0x7fc7380b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.480+0000 7fc737fff640 1 -- 172.21.15.181:0/3161093841 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7380b38a0 con 0x7fc7380718c0 2024-03-29T04:09:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.481+0000 7fc737fff640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc7380b2f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097a0 tx=0x7fc72802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90c6103d339248ad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.481+0000 7fc736ffd640 1 -- 172.21.15.181:0/3161093841 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728035020 con 0x7fc7380718c0 2024-03-29T04:09:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.481+0000 7fc736ffd640 1 -- 172.21.15.181:0/3161093841 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc728031830 con 0x7fc7380718c0 2024-03-29T04:09:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.481+0000 7fc736ffd640 1 -- 172.21.15.181:0/3161093841 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728031be0 con 0x7fc7380718c0 2024-03-29T04:09:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 -- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 msgr2=0x7fc7380b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc7380b2f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097a0 tx=0x7fc72802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 -- 172.21.15.181:0/3161093841 shutdown_connections 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc73815a710 0x7fc7380cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 --2- 172.21.15.181:0/3161093841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc7380b2f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 -- 172.21.15.181:0/3161093841 >> 172.21.15.181:0/3161093841 conn(0x7fc7380b5b00 msgr2=0x7fc73806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 -- 172.21.15.181:0/3161093841 shutdown_connections 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.482+0000 7fc73e7db640 1 -- 172.21.15.181:0/3161093841 wait complete. 2024-03-29T04:09:31.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.483+0000 7fc73e7db640 1 Processor -- start 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.483+0000 7fc73e7db640 1 -- start start 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc73815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7381597f0 0x7fc7381580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc738072c50 con 0x7fc73815a710 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7380b3e60 con 0x7fc7380718c0 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73e7db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7380d0490 con 0x7fc7381597f0 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34300/0 (socket says 172.21.15.181:34300) 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 -- 172.21.15.181:0/1945365917 learned_addr learned my addr 172.21.15.181:0/1945365917 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc73cd78640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7381597f0 0x7fc7381580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc737fff640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc73815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 -- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7381597f0 msgr2=0x7fc7381580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7381597f0 0x7fc7381580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 -- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 msgr2=0x7fc73815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc73815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc7377fe640 1 -- 172.21.15.181:0/1945365917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73815d180 con 0x7fc73815a710 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.484+0000 7fc737fff640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc73815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc7377fe640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e9b0 tx=0x7fc72c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c017070 con 0x7fc73815a710 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c00c9a0 con 0x7fc73815a710 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73815f770 con 0x7fc73815a710 2024-03-29T04:09:31.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00cd90 con 0x7fc73815a710 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.485+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc738158ff0 con 0x7fc73815a710 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.486+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc738075890 con 0x7fc73815a710 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.486+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc72c01a070 con 0x7fc73815a710 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.487+0000 7fc734ff9640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 0x7fc72406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.487+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc72c08bd10 con 0x7fc73815a710 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.487+0000 7fc737fff640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 0x7fc72406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.487+0000 7fc737fff640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 0x7fc72406e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc728036000 tx=0x7fc728008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:31.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.491+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc72c059980 con 0x7fc73815a710 2024-03-29T04:09:31.683 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:31.703+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc7380cc7f0 con 0x7fc73815a710 2024-03-29T04:09:32.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.357+0000 7fc734ff9640 1 -- 172.21.15.181:0/1945365917 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fc72c05dc80 con 0x7fc73815a710 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 msgr2=0x7fc72406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 0x7fc72406e9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc728036000 tx=0x7fc728008030 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 msgr2=0x7fc73815cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e9b0 tx=0x7fc72c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 shutdown_connections 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc72406c530 0x7fc72406e9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7381597f0 0x7fc7381580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc73815a710 0x7fc73815cc40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 --2- 172.21.15.181:0/1945365917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7380718c0 0x7fc73815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 >> 172.21.15.181:0/1945365917 conn(0x7fc7380b5b00 msgr2=0x7fc7380b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 shutdown_connections 2024-03-29T04:09:32.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.360+0000 7fc73e7db640 1 -- 172.21.15.181:0/1945365917 wait complete. 2024-03-29T04:09:32.352 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:09:32.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.476+0000 7fa7758b3640 1 Processor -- start 2024-03-29T04:09:32.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.476+0000 7fa7758b3640 1 -- start start 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa7758b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa77015e950 0x7fa7700d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa7758b3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7700d30d0 0x7fa7700b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa7758b3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7700d36f0 con 0x7fa7700d48c0 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa7758b3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa7700d3cb0 con 0x7fa7700d30d0 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa7758b3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa7700d4d00 con 0x7fa77015e950 2024-03-29T04:09:32.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76e7fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa7700d48c0 0x7fa7700d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57004/0 (socket says 172.21.15.181:57004) 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76e7fc640 1 -- 172.21.15.181:0/1933920818 learned_addr learned my addr 172.21.15.181:0/1933920818 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76f7fe640 1 --2- 172.21.15.181:0/1933920818 >> v2:172.21.15.181:3300/0 conn(0x7fa7700d30d0 0x7fa7700b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76effd640 1 --2- 172.21.15.181:0/1933920818 >> v2:172.21.15.187:3300/0 conn(0x7fa77015e950 0x7fa7700d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76dffb640 1 -- 172.21.15.181:0/1933920818 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109973246 0 0) 0x7fa7700d36f0 con 0x7fa7700d48c0 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76dffb640 1 -- 172.21.15.181:0/1933920818 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa758003520 con 0x7fa7700d48c0 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76f7fe640 1 -- 172.21.15.181:0/1933920818 >> v2:172.21.15.187:3300/0 conn(0x7fa77015e950 msgr2=0x7fa7700d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76f7fe640 1 --2- 172.21.15.181:0/1933920818 >> v2:172.21.15.187:3300/0 conn(0x7fa77015e950 0x7fa7700d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76f7fe640 1 -- 172.21.15.181:0/1933920818 >> v1:172.21.15.181:6789/0 conn(0x7fa7700d48c0 legacy=0x7fa7700d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.477+0000 7fa76f7fe640 1 -- 172.21.15.181:0/1933920818 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7700d39d0 con 0x7fa7700d30d0 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.478+0000 7fa76f7fe640 1 --2- 172.21.15.181:0/1933920818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d30d0 0x7fa7700b82c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7700d3e30 tx=0x7fa76802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf32eed2f5133a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.478+0000 7fa76dffb640 1 -- 172.21.15.181:0/1933920818 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768039070 con 0x7fa7700d30d0 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.478+0000 7fa76dffb640 1 -- 172.21.15.181:0/1933920818 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76802cd00 con 0x7fa7700d30d0 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.478+0000 7fa76dffb640 1 -- 172.21.15.181:0/1933920818 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768034660 con 0x7fa7700d30d0 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 -- 172.21.15.181:0/1933920818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d30d0 msgr2=0x7fa7700b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1933920818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d30d0 0x7fa7700b82c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7700d3e30 tx=0x7fa76802bf50 comp rx=0 tx=0).stop 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 -- 172.21.15.181:0/1933920818 shutdown_connections 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1933920818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d30d0 0x7fa7700b82c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1933920818 >> v2:172.21.15.187:3300/0 conn(0x7fa77015e950 0x7fa7700d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 -- 172.21.15.181:0/1933920818 >> 172.21.15.181:0/1933920818 conn(0x7fa7700676e0 msgr2=0x7fa770067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 -- 172.21.15.181:0/1933920818 shutdown_connections 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.479+0000 7fa7758b3640 1 -- 172.21.15.181:0/1933920818 wait complete. 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.480+0000 7fa7758b3640 1 Processor -- start 2024-03-29T04:09:32.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.480+0000 7fa7758b3640 1 -- start start 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7700d30d0 0x7fa7701506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa77015e950 0x7fa7701f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa77015ed30 con 0x7fa7700d48c0 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700b7da0 con 0x7fa77015e950 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa7758b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa77015eeb0 con 0x7fa7700d30d0 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34320/0 (socket says 172.21.15.181:34320) 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 -- 172.21.15.181:0/1111873085 learned_addr learned my addr 172.21.15.181:0/1111873085 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76f7fe640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa77015e950 0x7fa7701f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76effd640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7700d30d0 0x7fa7701506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 -- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7700d30d0 msgr2=0x7fa7701506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7700d30d0 0x7fa7701506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 -- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa77015e950 msgr2=0x7fa7701f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa77015e950 0x7fa7701f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.481+0000 7fa76e7fc640 1 -- 172.21.15.181:0/1111873085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77015fee0 con 0x7fa7700d48c0 2024-03-29T04:09:32.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa76e7fc640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c430 tx=0x7fa76400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764017070 con 0x7fa7700d48c0 2024-03-29T04:09:32.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa764005ce0 con 0x7fa7700d48c0 2024-03-29T04:09:32.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7701601c0 con 0x7fa7700d48c0 2024-03-29T04:09:32.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7640072f0 con 0x7fa7700d48c0 2024-03-29T04:09:32.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.482+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa770160780 con 0x7fa7700d48c0 2024-03-29T04:09:32.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.483+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770075a10 con 0x7fa7700d48c0 2024-03-29T04:09:32.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.483+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa76401a070 con 0x7fa7700d48c0 2024-03-29T04:09:32.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.483+0000 7fa757fff640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 0x7fa75c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.484+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa76408ae70 con 0x7fa7700d48c0 2024-03-29T04:09:32.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.484+0000 7fa76effd640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 0x7fa75c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.484+0000 7fa76effd640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 0x7fa75c06e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa760009aa0 tx=0x7fa760009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.488+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa764058b00 con 0x7fa7700d48c0 2024-03-29T04:09:32.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.698+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7700b8e00 con 0x7fa7700d48c0 2024-03-29T04:09:32.678 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.699+0000 7fa757fff640 1 -- 172.21.15.181:0/1111873085 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fa76405ce00 con 0x7fa7700d48c0 2024-03-29T04:09:32.679 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:32.679 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:32.679 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 msgr2=0x7fa75c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 0x7fa75c06e9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa760009aa0 tx=0x7fa760009190 comp rx=0 tx=0).stop 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 msgr2=0x7fa7701619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c430 tx=0x7fa76400c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 shutdown_connections 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa77015e950 0x7fa7701f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7700d48c0 0x7fa7701619a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa75c06c530 0x7fa75c06e9f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 --2- 172.21.15.181:0/1111873085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa7700d30d0 0x7fa7701506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 >> 172.21.15.181:0/1111873085 conn(0x7fa7700676e0 msgr2=0x7fa7700b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 shutdown_connections 2024-03-29T04:09:32.682 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.701+0000 7fa7758b3640 1 -- 172.21.15.181:0/1111873085 wait complete. 2024-03-29T04:09:32.692 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:32.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 Processor -- start 2024-03-29T04:09:32.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 -- start start 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 0x7f913c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f913c0d2f00 0x7f913c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913c0d36f0 con 0x7f913c15e950 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f913c0d3cb0 con 0x7f913c0d2f00 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f9141688640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0d39d0 con 0x7f913c0d4a90 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.813+0000 7f913b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f913c0d2f00 0x7f913c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913affd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f913c15e950 0x7f913c0d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57020/0 (socket says 172.21.15.181:57020) 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913affd640 1 -- 172.21.15.181:0/3551241898 learned_addr learned my addr 172.21.15.181:0/3551241898 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913a7fc640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 0x7f913c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913b7fe640 1 -- 172.21.15.181:0/3551241898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 msgr2=0x7f913c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913b7fe640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 0x7f913c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913b7fe640 1 -- 172.21.15.181:0/3551241898 >> v1:172.21.15.181:6789/0 conn(0x7f913c15e950 legacy=0x7f913c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913b7fe640 1 -- 172.21.15.181:0/3551241898 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c0b7ac0 con 0x7f913c0d2f00 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f9139ffb640 1 -- 172.21.15.181:0/3551241898 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621697847 0 0) 0x7f913c0d36f0 con 0x7f913c15e950 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.814+0000 7f913a7fc640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 0x7f913c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f913b7fe640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d2f00 0x7f913c0b82c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473644905ac6d144 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9139ffb640 1 -- 172.21.15.181:0/3551241898 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c039070 con 0x7f913c0d2f00 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9139ffb640 1 -- 172.21.15.181:0/3551241898 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c02cce0 con 0x7f913c0d2f00 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9139ffb640 1 -- 172.21.15.181:0/3551241898 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c034660 con 0x7f913c0d2f00 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 -- 172.21.15.181:0/3551241898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d2f00 msgr2=0x7f913c0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d2f00 0x7f913c0b82c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02bf30 comp rx=0 tx=0).stop 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 -- 172.21.15.181:0/3551241898 shutdown_connections 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d2f00 0x7f913c0b82c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 --2- 172.21.15.181:0/3551241898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d4a90 0x7f913c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 -- 172.21.15.181:0/3551241898 >> 172.21.15.181:0/3551241898 conn(0x7f913c0676e0 msgr2=0x7f913c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 -- 172.21.15.181:0/3551241898 shutdown_connections 2024-03-29T04:09:32.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.815+0000 7f9141688640 1 -- 172.21.15.181:0/3551241898 wait complete. 2024-03-29T04:09:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.816+0000 7f9141688640 1 Processor -- start 2024-03-29T04:09:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 -- start start 2024-03-29T04:09:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d2f00 0x7f913c14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f913c15e950 0x7f913c0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f913c15ed30 con 0x7f913c0d4a90 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0d33b0 con 0x7f913c15e950 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f9141688640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f913c15eeb0 con 0x7f913c0d2f00 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f913a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f913a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34340/0 (socket says 172.21.15.181:34340) 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f913a7fc640 1 -- 172.21.15.181:0/2569918363 learned_addr learned my addr 172.21.15.181:0/2569918363 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f913b7fe640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f913c15e950 0x7f913c0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.817+0000 7f913affd640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d2f00 0x7f913c14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 -- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d2f00 msgr2=0x7f913c14cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d2f00 0x7f913c14cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 -- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f913c15e950 msgr2=0x7f913c0c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f913c15e950 0x7f913c0c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 -- 172.21.15.181:0/2569918363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c160a50 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f913a7fc640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f913000ebe0 tx=0x7f913000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130017070 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913000cc30 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c15f9b0 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130013470 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.818+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c15ff70 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.819+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c075a10 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.819+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f913001a070 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.820+0000 7f91237fe640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 0x7f912806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.820+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f913008b890 con 0x7f913c0d4a90 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.820+0000 7f913affd640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 0x7f912806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:32.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.820+0000 7f913affd640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 0x7f912806e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:32.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.824+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9130059520 con 0x7f913c0d4a90 2024-03-29T04:09:32.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:32.999+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f913c15dca0 con 0x7f913c0d4a90 2024-03-29T04:09:32.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.001+0000 7f91237fe640 1 -- 172.21.15.181:0/2569918363 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f913005d820 con 0x7f913c0d4a90 2024-03-29T04:09:32.980 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:32.981 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 msgr2=0x7f912806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 0x7f912806e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134007de0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 msgr2=0x7f913c15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f913000ebe0 tx=0x7f913000c520 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 shutdown_connections 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f913c15e950 0x7f913c0c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f913c0d4a90 0x7f913c15f0b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f912806c530 0x7f912806e9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 --2- 172.21.15.181:0/2569918363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f913c0d2f00 0x7f913c14cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 >> 172.21.15.181:0/2569918363 conn(0x7f913c0676e0 msgr2=0x7f913c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 shutdown_connections 2024-03-29T04:09:32.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.003+0000 7f9141688640 1 -- 172.21.15.181:0/2569918363 wait complete. 2024-03-29T04:09:32.993 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:09:32.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:33.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.114+0000 7f87dc6b8640 1 Processor -- start 2024-03-29T04:09:33.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.114+0000 7f87dc6b8640 1 -- start start 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dc6b8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 0x7f87d40ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dc6b8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87d4072230 0x7f87d4072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dc6b8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d4072b80 con 0x7f87d415c220 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dc6b8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f87d415c600 con 0x7f87d4072230 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dc6b8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f87d406bdd0 con 0x7f87d40d0360 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dac55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87d4072230 0x7f87d4072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87da454640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f87d415c220 0x7f87d406b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57026/0 (socket says 172.21.15.181:57026) 2024-03-29T04:09:33.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87da454640 1 -- 172.21.15.181:0/1400545190 learned_addr learned my addr 172.21.15.181:0/1400545190 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87d9c53640 1 --2- 172.21.15.181:0/1400545190 >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 0x7f87d40ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dac55640 1 -- 172.21.15.181:0/1400545190 >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 msgr2=0x7f87d40ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dac55640 1 --2- 172.21.15.181:0/1400545190 >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 0x7f87d40ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dac55640 1 -- 172.21.15.181:0/1400545190 >> v1:172.21.15.181:6789/0 conn(0x7f87d415c220 legacy=0x7f87d406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87d9452640 1 -- 172.21.15.181:0/1400545190 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471998268 0 0) 0x7f87d0001110 con 0x7f87d415c220 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87dac55640 1 -- 172.21.15.181:0/1400545190 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d406c0b0 con 0x7f87d4072230 2024-03-29T04:09:33.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.115+0000 7f87d9c53640 1 --2- 172.21.15.181:0/1400545190 >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 0x7f87d40ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:33.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.116+0000 7f87dac55640 1 --2- 172.21.15.181:0/1400545190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d4072230 0x7f87d4072640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87d4071d20 tx=0x7f87cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758e84e4e9355a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.116+0000 7f87d9452640 1 -- 172.21.15.181:0/1400545190 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc039070 con 0x7f87d4072230 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.116+0000 7f87d9452640 1 -- 172.21.15.181:0/1400545190 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc02cc80 con 0x7f87d4072230 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.116+0000 7f87d9452640 1 -- 172.21.15.181:0/1400545190 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc034660 con 0x7f87d4072230 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 -- 172.21.15.181:0/1400545190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d4072230 msgr2=0x7f87d4072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/1400545190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d4072230 0x7f87d4072640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87d4071d20 tx=0x7f87cc02bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 -- 172.21.15.181:0/1400545190 shutdown_connections 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/1400545190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d4072230 0x7f87d4072640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/1400545190 >> v2:172.21.15.187:3300/0 conn(0x7f87d40d0360 0x7f87d40ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 -- 172.21.15.181:0/1400545190 >> 172.21.15.181:0/1400545190 conn(0x7f87d40d0cb0 msgr2=0x7f87d40d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 -- 172.21.15.181:0/1400545190 shutdown_connections 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.117+0000 7f87dc6b8640 1 -- 172.21.15.181:0/1400545190 wait complete. 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.118+0000 7f87dc6b8640 1 Processor -- start 2024-03-29T04:09:33.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.118+0000 7f87dc6b8640 1 -- start start 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d4072230 0x7f87d41461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d415c220 0x7f87d41536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87d40b5630 con 0x7f87d40d0360 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87d40b5980 con 0x7f87d415c220 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87d40ce9a0 con 0x7f87d4072230 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34348/0 (socket says 172.21.15.181:34348) 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 -- 172.21.15.181:0/4209453466 learned_addr learned my addr 172.21.15.181:0/4209453466 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87da454640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d4072230 0x7f87d41461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dac55640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d415c220 0x7f87d41536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 -- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d4072230 msgr2=0x7f87d41461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d4072230 0x7f87d41461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 -- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d415c220 msgr2=0x7f87d41536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d415c220 0x7f87d41536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 -- 172.21.15.181:0/4209453466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d41e6270 con 0x7f87d40d0360 2024-03-29T04:09:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87d9c53640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e960 tx=0x7f87c400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4017070 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c400ca00 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d4155c70 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c400cdd0 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.119+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d4146720 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.120+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4156060 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.121+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f87c401a070 con 0x7f87d40d0360 2024-03-29T04:09:33.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.121+0000 7f87caffd640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 0x7f87b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.121+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f87c408b860 con 0x7f87d40d0360 2024-03-29T04:09:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.121+0000 7f87da454640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 0x7f87b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.122+0000 7f87da454640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 0x7f87b806e9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007fd0 tx=0x7f87d0007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.126+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f87c40594f0 con 0x7f87d40d0360 2024-03-29T04:09:33.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.300+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87d4075a10 con 0x7f87b806c530 2024-03-29T04:09:33.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.309+0000 7f87caffd640 1 -- 172.21.15.181:0/4209453466 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f87d4075a10 con 0x7f87b806c530 2024-03-29T04:09:33.292 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:33.295 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 msgr2=0x7f87b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 0x7f87b806e9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007fd0 tx=0x7f87d0007de0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 msgr2=0x7f87d41e5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e960 tx=0x7f87c400ee20 comp rx=0 tx=0).stop 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 shutdown_connections 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d415c220 0x7f87d41536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d40d0360 0x7f87d41e5860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f87b806c530 0x7f87b806e9f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 --2- 172.21.15.181:0/4209453466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87d4072230 0x7f87d41461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 >> 172.21.15.181:0/4209453466 conn(0x7f87d40d0cb0 msgr2=0x7f87d40b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 shutdown_connections 2024-03-29T04:09:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.312+0000 7f87dc6b8640 1 -- 172.21.15.181:0/4209453466 wait complete. 2024-03-29T04:09:33.302 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-03-29T04:09:31.576790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2048878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001841"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656335+0000","last_change":"2024-03-29T04:09:15.308724+0000","last_active":"2024-03-29T04:09:22.656335+0000","last_peered":"2024-03-29T04:09:22.656335+0000","last_clean":"2024-03-29T04:09:22.656335+0000","last_became_active":"2024-03-29T04:09:15.308445+0000","last_became_peered":"2024-03-29T04:09:15.308445+0000","last_unstale":"2024-03-29T04:09:22.656335+0000","last_undegraded":"2024-03-29T04:09:22.656335+0000","last_fullsized":"2024-03-29T04:09:22.656335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329448+0000","last_change":"2024-03-29T04:09:16.307371+0000","last_active":"2024-03-29T04:09:22.329448+0000","last_peered":"2024-03-29T04:09:22.329448+0000","last_clean":"2024-03-29T04:09:22.329448+0000","last_became_active":"2024-03-29T04:09:16.307077+0000","last_became_peered":"2024-03-29T04:09:16.307077+0000","last_unstale":"2024-03-29T04:09:22.329448+0000","last_undegraded":"2024-03-29T04:09:22.329448+0000","last_fullsized":"2024-03-29T04:09:22.329448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328852+0000","last_change":"2024-03-29T04:09:20.883785+0000","last_active":"2024-03-29T04:09:22.328852+0000","last_peered":"2024-03-29T04:09:22.328852+0000","last_clean":"2024-03-29T04:09:22.328852+0000","last_became_active":"2024-03-29T04:09:20.883360+0000","last_became_peered":"2024-03-29T04:09:20.883360+0000","last_unstale":"2024-03-29T04:09:22.328852+0000","last_undegraded":"2024-03-29T04:09:22.328852+0000","last_fullsized":"2024-03-29T04:09:22.328852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.908030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329590+0000","last_change":"2024-03-29T04:09:15.301577+0000","last_active":"2024-03-29T04:09:22.329590+0000","last_peered":"2024-03-29T04:09:22.329590+0000","last_clean":"2024-03-29T04:09:22.329590+0000","last_became_active":"2024-03-29T04:09:15.301247+0000","last_became_peered":"2024-03-29T04:09:15.301247+0000","last_unstale":"2024-03-29T04:09:22.329590+0000","last_undegraded":"2024-03-29T04:09:22.329590+0000","last_fullsized":"2024-03-29T04:09:22.329590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656042+0000","last_change":"2024-03-29T04:09:16.319168+0000","last_active":"2024-03-29T04:09:22.656042+0000","last_peered":"2024-03-29T04:09:22.656042+0000","last_clean":"2024-03-29T04:09:22.656042+0000","last_became_active":"2024-03-29T04:09:16.318673+0000","last_became_peered":"2024-03-29T04:09:16.318673+0000","last_unstale":"2024-03-29T04:09:22.656042+0000","last_undegraded":"2024-03-29T04:09:22.656042+0000","last_fullsized":"2024-03-29T04:09:22.656042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:41.049934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329629+0000","last_change":"2024-03-29T04:09:20.348184+0000","last_active":"2024-03-29T04:09:22.329629+0000","last_peered":"2024-03-29T04:09:22.329629+0000","last_clean":"2024-03-29T04:09:22.329629+0000","last_became_active":"2024-03-29T04:09:20.347504+0000","last_became_peered":"2024-03-29T04:09:20.347504+0000","last_unstale":"2024-03-29T04:09:22.329629+0000","last_undegraded":"2024-03-29T04:09:22.329629+0000","last_fullsized":"2024-03-29T04:09:22.329629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656067+0000","last_change":"2024-03-29T04:09:15.308597+0000","last_active":"2024-03-29T04:09:22.656067+0000","last_peered":"2024-03-29T04:09:22.656067+0000","last_clean":"2024-03-29T04:09:22.656067+0000","last_became_active":"2024-03-29T04:09:15.308020+0000","last_became_peered":"2024-03-29T04:09:15.308020+0000","last_unstale":"2024-03-29T04:09:22.656067+0000","last_undegraded":"2024-03-29T04:09:22.656067+0000","last_fullsized":"2024-03-29T04:09:22.656067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:40.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655201+0000","last_change":"2024-03-29T04:09:16.321774+0000","last_active":"2024-03-29T04:09:22.655201+0000","last_peered":"2024-03-29T04:09:22.655201+0000","last_clean":"2024-03-29T04:09:22.655201+0000","last_became_active":"2024-03-29T04:09:16.321464+0000","last_became_peered":"2024-03-29T04:09:16.321464+0000","last_unstale":"2024-03-29T04:09:22.655201+0000","last_undegraded":"2024-03-29T04:09:22.655201+0000","last_fullsized":"2024-03-29T04:09:22.655201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330320+0000","last_change":"2024-03-29T04:09:20.346472+0000","last_active":"2024-03-29T04:09:22.330320+0000","last_peered":"2024-03-29T04:09:22.330320+0000","last_clean":"2024-03-29T04:09:22.330320+0000","last_became_active":"2024-03-29T04:09:20.346130+0000","last_became_peered":"2024-03-29T04:09:20.346130+0000","last_unstale":"2024-03-29T04:09:22.330320+0000","last_undegraded":"2024-03-29T04:09:22.330320+0000","last_fullsized":"2024-03-29T04:09:22.330320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655700+0000","last_change":"2024-03-29T04:09:15.308906+0000","last_active":"2024-03-29T04:09:22.655700+0000","last_peered":"2024-03-29T04:09:22.655700+0000","last_clean":"2024-03-29T04:09:22.655700+0000","last_became_active":"2024-03-29T04:09:15.308632+0000","last_became_peered":"2024-03-29T04:09:15.308632+0000","last_unstale":"2024-03-29T04:09:22.655700+0000","last_undegraded":"2024-03-29T04:09:22.655700+0000","last_fullsized":"2024-03-29T04:09:22.655700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.323173+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.322174+0000","last_became_peered":"2024-03-29T04:09:16.322174+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329721+0000","last_change":"2024-03-29T04:09:20.341688+0000","last_active":"2024-03-29T04:09:22.329721+0000","last_peered":"2024-03-29T04:09:22.329721+0000","last_clean":"2024-03-29T04:09:22.329721+0000","last_became_active":"2024-03-29T04:09:20.341328+0000","last_became_peered":"2024-03-29T04:09:20.341328+0000","last_unstale":"2024-03-29T04:09:22.329721+0000","last_undegraded":"2024-03-29T04:09:22.329721+0000","last_fullsized":"2024-03-29T04:09:22.329721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329443+0000","last_change":"2024-03-29T04:09:16.318229+0000","last_active":"2024-03-29T04:09:22.329443+0000","last_peered":"2024-03-29T04:09:22.329443+0000","last_clean":"2024-03-29T04:09:22.329443+0000","last_became_active":"2024-03-29T04:09:16.317967+0000","last_became_peered":"2024-03-29T04:09:16.317967+0000","last_unstale":"2024-03-29T04:09:22.329443+0000","last_undegraded":"2024-03-29T04:09:22.329443+0000","last_fullsized":"2024-03-29T04:09:22.329443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655679+0000","last_change":"2024-03-29T04:09:15.308430+0000","last_active":"2024-03-29T04:09:22.655679+0000","last_peered":"2024-03-29T04:09:22.655679+0000","last_clean":"2024-03-29T04:09:22.655679+0000","last_became_active":"2024-03-29T04:09:15.308131+0000","last_became_peered":"2024-03-29T04:09:15.308131+0000","last_unstale":"2024-03-29T04:09:22.655679+0000","last_undegraded":"2024-03-29T04:09:22.655679+0000","last_fullsized":"2024-03-29T04:09:22.655679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:36:32.401409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329454+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.329454+0000","last_peered":"2024-03-29T04:09:22.329454+0000","last_clean":"2024-03-29T04:09:22.329454+0000","last_became_active":"2024-03-29T04:09:20.347330+0000","last_became_peered":"2024-03-29T04:09:20.347330+0000","last_unstale":"2024-03-29T04:09:22.329454+0000","last_undegraded":"2024-03-29T04:09:22.329454+0000","last_fullsized":"2024-03-29T04:09:22.329454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329091+0000","last_change":"2024-03-29T04:09:16.311370+0000","last_active":"2024-03-29T04:09:22.329091+0000","last_peered":"2024-03-29T04:09:22.329091+0000","last_clean":"2024-03-29T04:09:22.329091+0000","last_became_active":"2024-03-29T04:09:16.311106+0000","last_became_peered":"2024-03-29T04:09:16.311106+0000","last_unstale":"2024-03-29T04:09:22.329091+0000","last_undegraded":"2024-03-29T04:09:22.329091+0000","last_fullsized":"2024-03-29T04:09:22.329091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656398+0000","last_change":"2024-03-29T04:09:15.306878+0000","last_active":"2024-03-29T04:09:22.656398+0000","last_peered":"2024-03-29T04:09:22.656398+0000","last_clean":"2024-03-29T04:09:22.656398+0000","last_became_active":"2024-03-29T04:09:15.306590+0000","last_became_peered":"2024-03-29T04:09:15.306590+0000","last_unstale":"2024-03-29T04:09:22.656398+0000","last_undegraded":"2024-03-29T04:09:22.656398+0000","last_fullsized":"2024-03-29T04:09:22.656398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330041+0000","last_change":"2024-03-29T04:09:20.347808+0000","last_active":"2024-03-29T04:09:22.330041+0000","last_peered":"2024-03-29T04:09:22.330041+0000","last_clean":"2024-03-29T04:09:22.330041+0000","last_became_active":"2024-03-29T04:09:20.347419+0000","last_became_peered":"2024-03-29T04:09:20.347419+0000","last_unstale":"2024-03-29T04:09:22.330041+0000","last_undegraded":"2024-03-29T04:09:22.330041+0000","last_fullsized":"2024-03-29T04:09:22.330041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:58.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328945+0000","last_change":"2024-03-29T04:09:16.309708+0000","last_active":"2024-03-29T04:09:22.328945+0000","last_peered":"2024-03-29T04:09:22.328945+0000","last_clean":"2024-03-29T04:09:22.328945+0000","last_became_active":"2024-03-29T04:09:16.309388+0000","last_became_peered":"2024-03-29T04:09:16.309388+0000","last_unstale":"2024-03-29T04:09:22.328945+0000","last_undegraded":"2024-03-29T04:09:22.328945+0000","last_fullsized":"2024-03-29T04:09:22.328945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655404+0000","last_change":"2024-03-29T04:09:15.308764+0000","last_active":"2024-03-29T04:09:22.655404+0000","last_peered":"2024-03-29T04:09:22.655404+0000","last_clean":"2024-03-29T04:09:22.655404+0000","last_became_active":"2024-03-29T04:09:15.308466+0000","last_became_peered":"2024-03-29T04:09:15.308466+0000","last_unstale":"2024-03-29T04:09:22.655404+0000","last_undegraded":"2024-03-29T04:09:22.655404+0000","last_fullsized":"2024-03-29T04:09:22.655404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657055+0000","last_change":"2024-03-29T04:09:20.883567+0000","last_active":"2024-03-29T04:09:22.657055+0000","last_peered":"2024-03-29T04:09:22.657055+0000","last_clean":"2024-03-29T04:09:22.657055+0000","last_became_active":"2024-03-29T04:09:20.883215+0000","last_became_peered":"2024-03-29T04:09:20.883215+0000","last_unstale":"2024-03-29T04:09:22.657055+0000","last_undegraded":"2024-03-29T04:09:22.657055+0000","last_fullsized":"2024-03-29T04:09:22.657055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330723+0000","last_change":"2024-03-29T04:09:16.307731+0000","last_active":"2024-03-29T04:09:22.330723+0000","last_peered":"2024-03-29T04:09:22.330723+0000","last_clean":"2024-03-29T04:09:22.330723+0000","last_became_active":"2024-03-29T04:09:16.307458+0000","last_became_peered":"2024-03-29T04:09:16.307458+0000","last_unstale":"2024-03-29T04:09:22.330723+0000","last_undegraded":"2024-03-29T04:09:22.330723+0000","last_fullsized":"2024-03-29T04:09:22.330723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:43.534030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654990+0000","last_change":"2024-03-29T04:09:15.306750+0000","last_active":"2024-03-29T04:09:22.654990+0000","last_peered":"2024-03-29T04:09:22.654990+0000","last_clean":"2024-03-29T04:09:22.654990+0000","last_became_active":"2024-03-29T04:09:15.306458+0000","last_became_peered":"2024-03-29T04:09:15.306458+0000","last_unstale":"2024-03-29T04:09:22.654990+0000","last_undegraded":"2024-03-29T04:09:22.654990+0000","last_fullsized":"2024-03-29T04:09:22.654990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330722+0000","last_change":"2024-03-29T04:09:20.346709+0000","last_active":"2024-03-29T04:09:22.330722+0000","last_peered":"2024-03-29T04:09:22.330722+0000","last_clean":"2024-03-29T04:09:22.330722+0000","last_became_active":"2024-03-29T04:09:20.346366+0000","last_became_peered":"2024-03-29T04:09:20.346366+0000","last_unstale":"2024-03-29T04:09:22.330722+0000","last_undegraded":"2024-03-29T04:09:22.330722+0000","last_fullsized":"2024-03-29T04:09:22.330722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656086+0000","last_change":"2024-03-29T04:09:16.311388+0000","last_active":"2024-03-29T04:09:22.656086+0000","last_peered":"2024-03-29T04:09:22.656086+0000","last_clean":"2024-03-29T04:09:22.656086+0000","last_became_active":"2024-03-29T04:09:16.311119+0000","last_became_peered":"2024-03-29T04:09:16.311119+0000","last_unstale":"2024-03-29T04:09:22.656086+0000","last_undegraded":"2024-03-29T04:09:22.656086+0000","last_fullsized":"2024-03-29T04:09:22.656086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330110+0000","last_change":"2024-03-29T04:09:15.301237+0000","last_active":"2024-03-29T04:09:22.330110+0000","last_peered":"2024-03-29T04:09:22.330110+0000","last_clean":"2024-03-29T04:09:22.330110+0000","last_became_active":"2024-03-29T04:09:15.300671+0000","last_became_peered":"2024-03-29T04:09:15.300671+0000","last_unstale":"2024-03-29T04:09:22.330110+0000","last_undegraded":"2024-03-29T04:09:22.330110+0000","last_fullsized":"2024-03-29T04:09:22.330110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330650+0000","last_change":"2024-03-29T04:09:20.882096+0000","last_active":"2024-03-29T04:09:22.330650+0000","last_peered":"2024-03-29T04:09:22.330650+0000","last_clean":"2024-03-29T04:09:22.330650+0000","last_became_active":"2024-03-29T04:09:20.881672+0000","last_became_peered":"2024-03-29T04:09:20.881672+0000","last_unstale":"2024-03-29T04:09:22.330650+0000","last_undegraded":"2024-03-29T04:09:22.330650+0000","last_fullsized":"2024-03-29T04:09:22.330650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329679+0000","last_change":"2024-03-29T04:09:16.308204+0000","last_active":"2024-03-29T04:09:22.329679+0000","last_peered":"2024-03-29T04:09:22.329679+0000","last_clean":"2024-03-29T04:09:22.329679+0000","last_became_active":"2024-03-29T04:09:16.307931+0000","last_became_peered":"2024-03-29T04:09:16.307931+0000","last_unstale":"2024-03-29T04:09:22.329679+0000","last_undegraded":"2024-03-29T04:09:22.329679+0000","last_fullsized":"2024-03-29T04:09:22.329679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656940+0000","last_change":"2024-03-29T04:09:15.303590+0000","last_active":"2024-03-29T04:09:22.656940+0000","last_peered":"2024-03-29T04:09:22.656940+0000","last_clean":"2024-03-29T04:09:22.656940+0000","last_became_active":"2024-03-29T04:09:15.303295+0000","last_became_peered":"2024-03-29T04:09:15.303295+0000","last_unstale":"2024-03-29T04:09:22.656940+0000","last_undegraded":"2024-03-29T04:09:22.656940+0000","last_fullsized":"2024-03-29T04:09:22.656940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328789+0000","last_change":"2024-03-29T04:09:20.884166+0000","last_active":"2024-03-29T04:09:22.328789+0000","last_peered":"2024-03-29T04:09:22.328789+0000","last_clean":"2024-03-29T04:09:22.328789+0000","last_became_active":"2024-03-29T04:09:20.883801+0000","last_became_peered":"2024-03-29T04:09:20.883801+0000","last_unstale":"2024-03-29T04:09:22.328789+0000","last_undegraded":"2024-03-29T04:09:22.328789+0000","last_fullsized":"2024-03-29T04:09:22.328789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:45.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656397+0000","last_change":"2024-03-29T04:09:16.307299+0000","last_active":"2024-03-29T04:09:22.656397+0000","last_peered":"2024-03-29T04:09:22.656397+0000","last_clean":"2024-03-29T04:09:22.656397+0000","last_became_active":"2024-03-29T04:09:16.306982+0000","last_became_peered":"2024-03-29T04:09:16.306982+0000","last_unstale":"2024-03-29T04:09:22.656397+0000","last_undegraded":"2024-03-29T04:09:22.656397+0000","last_fullsized":"2024-03-29T04:09:22.656397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656158+0000","last_change":"2024-03-29T04:09:15.308823+0000","last_active":"2024-03-29T04:09:22.656158+0000","last_peered":"2024-03-29T04:09:22.656158+0000","last_clean":"2024-03-29T04:09:22.656158+0000","last_became_active":"2024-03-29T04:09:15.308335+0000","last_became_peered":"2024-03-29T04:09:15.308335+0000","last_unstale":"2024-03-29T04:09:22.656158+0000","last_undegraded":"2024-03-29T04:09:22.656158+0000","last_fullsized":"2024-03-29T04:09:22.656158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:09.173137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655324+0000","last_change":"2024-03-29T04:09:20.882760+0000","last_active":"2024-03-29T04:09:22.655324+0000","last_peered":"2024-03-29T04:09:22.655324+0000","last_clean":"2024-03-29T04:09:22.655324+0000","last_became_active":"2024-03-29T04:09:20.881819+0000","last_became_peered":"2024-03-29T04:09:20.881819+0000","last_unstale":"2024-03-29T04:09:22.655324+0000","last_undegraded":"2024-03-29T04:09:22.655324+0000","last_fullsized":"2024-03-29T04:09:22.655324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655417+0000","last_change":"2024-03-29T04:09:16.318039+0000","last_active":"2024-03-29T04:09:22.655417+0000","last_peered":"2024-03-29T04:09:22.655417+0000","last_clean":"2024-03-29T04:09:22.655417+0000","last_became_active":"2024-03-29T04:09:16.317784+0000","last_became_peered":"2024-03-29T04:09:16.317784+0000","last_unstale":"2024-03-29T04:09:22.655417+0000","last_undegraded":"2024-03-29T04:09:22.655417+0000","last_fullsized":"2024-03-29T04:09:22.655417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656459+0000","last_change":"2024-03-29T04:09:15.302818+0000","last_active":"2024-03-29T04:09:22.656459+0000","last_peered":"2024-03-29T04:09:22.656459+0000","last_clean":"2024-03-29T04:09:22.656459+0000","last_became_active":"2024-03-29T04:09:15.302521+0000","last_became_peered":"2024-03-29T04:09:15.302521+0000","last_unstale":"2024-03-29T04:09:22.656459+0000","last_undegraded":"2024-03-29T04:09:22.656459+0000","last_fullsized":"2024-03-29T04:09:22.656459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656072+0000","last_change":"2024-03-29T04:09:20.351951+0000","last_active":"2024-03-29T04:09:22.656072+0000","last_peered":"2024-03-29T04:09:22.656072+0000","last_clean":"2024-03-29T04:09:22.656072+0000","last_became_active":"2024-03-29T04:09:20.351337+0000","last_became_peered":"2024-03-29T04:09:20.351337+0000","last_unstale":"2024-03-29T04:09:22.656072+0000","last_undegraded":"2024-03-29T04:09:22.656072+0000","last_fullsized":"2024-03-29T04:09:22.656072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330495+0000","last_change":"2024-03-29T04:09:16.316074+0000","last_active":"2024-03-29T04:09:22.330495+0000","last_peered":"2024-03-29T04:09:22.330495+0000","last_clean":"2024-03-29T04:09:22.330495+0000","last_became_active":"2024-03-29T04:09:16.315644+0000","last_became_peered":"2024-03-29T04:09:16.315644+0000","last_unstale":"2024-03-29T04:09:22.330495+0000","last_undegraded":"2024-03-29T04:09:22.330495+0000","last_fullsized":"2024-03-29T04:09:22.330495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:13.041449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655320+0000","last_change":"2024-03-29T04:09:15.307707+0000","last_active":"2024-03-29T04:09:22.655320+0000","last_peered":"2024-03-29T04:09:22.655320+0000","last_clean":"2024-03-29T04:09:22.655320+0000","last_became_active":"2024-03-29T04:09:15.307411+0000","last_became_peered":"2024-03-29T04:09:15.307411+0000","last_unstale":"2024-03-29T04:09:22.655320+0000","last_undegraded":"2024-03-29T04:09:22.655320+0000","last_fullsized":"2024-03-29T04:09:22.655320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656552+0000","last_change":"2024-03-29T04:09:20.346773+0000","last_active":"2024-03-29T04:09:22.656552+0000","last_peered":"2024-03-29T04:09:22.656552+0000","last_clean":"2024-03-29T04:09:22.656552+0000","last_became_active":"2024-03-29T04:09:20.346383+0000","last_became_peered":"2024-03-29T04:09:20.346383+0000","last_unstale":"2024-03-29T04:09:22.656552+0000","last_undegraded":"2024-03-29T04:09:22.656552+0000","last_fullsized":"2024-03-29T04:09:22.656552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.311447+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.311171+0000","last_became_peered":"2024-03-29T04:09:16.311171+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656334+0000","last_change":"2024-03-29T04:09:15.307046+0000","last_active":"2024-03-29T04:09:22.656334+0000","last_peered":"2024-03-29T04:09:22.656334+0000","last_clean":"2024-03-29T04:09:22.656334+0000","last_became_active":"2024-03-29T04:09:15.305761+0000","last_became_peered":"2024-03-29T04:09:15.305761+0000","last_unstale":"2024-03-29T04:09:22.656334+0000","last_undegraded":"2024-03-29T04:09:22.656334+0000","last_fullsized":"2024-03-29T04:09:22.656334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657098+0000","last_change":"2024-03-29T04:09:20.346732+0000","last_active":"2024-03-29T04:09:22.657098+0000","last_peered":"2024-03-29T04:09:22.657098+0000","last_clean":"2024-03-29T04:09:22.657098+0000","last_became_active":"2024-03-29T04:09:20.346121+0000","last_became_peered":"2024-03-29T04:09:20.346121+0000","last_unstale":"2024-03-29T04:09:22.657098+0000","last_undegraded":"2024-03-29T04:09:22.657098+0000","last_fullsized":"2024-03-29T04:09:22.657098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655371+0000","last_change":"2024-03-29T04:09:16.318196+0000","last_active":"2024-03-29T04:09:22.655371+0000","last_peered":"2024-03-29T04:09:22.655371+0000","last_clean":"2024-03-29T04:09:22.655371+0000","last_became_active":"2024-03-29T04:09:16.317906+0000","last_became_peered":"2024-03-29T04:09:16.317906+0000","last_unstale":"2024-03-29T04:09:22.655371+0000","last_undegraded":"2024-03-29T04:09:22.655371+0000","last_fullsized":"2024-03-29T04:09:22.655371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:36.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655070+0000","last_change":"2024-03-29T04:09:15.308083+0000","last_active":"2024-03-29T04:09:22.655070+0000","last_peered":"2024-03-29T04:09:22.655070+0000","last_clean":"2024-03-29T04:09:22.655070+0000","last_became_active":"2024-03-29T04:09:15.307813+0000","last_became_peered":"2024-03-29T04:09:15.307813+0000","last_unstale":"2024-03-29T04:09:22.655070+0000","last_undegraded":"2024-03-29T04:09:22.655070+0000","last_fullsized":"2024-03-29T04:09:22.655070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655909+0000","last_change":"2024-03-29T04:09:20.882216+0000","last_active":"2024-03-29T04:09:22.655909+0000","last_peered":"2024-03-29T04:09:22.655909+0000","last_clean":"2024-03-29T04:09:22.655909+0000","last_became_active":"2024-03-29T04:09:20.881801+0000","last_became_peered":"2024-03-29T04:09:20.881801+0000","last_unstale":"2024-03-29T04:09:22.655909+0000","last_undegraded":"2024-03-29T04:09:22.655909+0000","last_fullsized":"2024-03-29T04:09:22.655909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328920+0000","last_change":"2024-03-29T04:09:16.319621+0000","last_active":"2024-03-29T04:09:22.328920+0000","last_peered":"2024-03-29T04:09:22.328920+0000","last_clean":"2024-03-29T04:09:22.328920+0000","last_became_active":"2024-03-29T04:09:16.318978+0000","last_became_peered":"2024-03-29T04:09:16.318978+0000","last_unstale":"2024-03-29T04:09:22.328920+0000","last_undegraded":"2024-03-29T04:09:22.328920+0000","last_fullsized":"2024-03-29T04:09:22.328920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655084+0000","last_change":"2024-03-29T04:09:15.302638+0000","last_active":"2024-03-29T04:09:22.655084+0000","last_peered":"2024-03-29T04:09:22.655084+0000","last_clean":"2024-03-29T04:09:22.655084+0000","last_became_active":"2024-03-29T04:09:15.302211+0000","last_became_peered":"2024-03-29T04:09:15.302211+0000","last_unstale":"2024-03-29T04:09:22.655084+0000","last_undegraded":"2024-03-29T04:09:22.655084+0000","last_fullsized":"2024-03-29T04:09:22.655084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655145+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:09:22.655145+0000","last_peered":"2024-03-29T04:09:22.655145+0000","last_clean":"2024-03-29T04:09:22.655145+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:09:22.655145+0000","last_undegraded":"2024-03-29T04:09:22.655145+0000","last_fullsized":"2024-03-29T04:09:22.655145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329351+0000","last_change":"2024-03-29T04:09:20.883146+0000","last_active":"2024-03-29T04:09:22.329351+0000","last_peered":"2024-03-29T04:09:22.329351+0000","last_clean":"2024-03-29T04:09:22.329351+0000","last_became_active":"2024-03-29T04:09:20.882795+0000","last_became_peered":"2024-03-29T04:09:20.882795+0000","last_unstale":"2024-03-29T04:09:22.329351+0000","last_undegraded":"2024-03-29T04:09:22.329351+0000","last_fullsized":"2024-03-29T04:09:22.329351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655937+0000","last_change":"2024-03-29T04:09:16.316245+0000","last_active":"2024-03-29T04:09:22.655937+0000","last_peered":"2024-03-29T04:09:22.655937+0000","last_clean":"2024-03-29T04:09:22.655937+0000","last_became_active":"2024-03-29T04:09:16.315972+0000","last_became_peered":"2024-03-29T04:09:16.315972+0000","last_unstale":"2024-03-29T04:09:22.655937+0000","last_undegraded":"2024-03-29T04:09:22.655937+0000","last_fullsized":"2024-03-29T04:09:22.655937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656723+0000","last_change":"2024-03-29T04:09:15.304758+0000","last_active":"2024-03-29T04:09:22.656723+0000","last_peered":"2024-03-29T04:09:22.656723+0000","last_clean":"2024-03-29T04:09:22.656723+0000","last_became_active":"2024-03-29T04:09:15.304450+0000","last_became_peered":"2024-03-29T04:09:15.304450+0000","last_unstale":"2024-03-29T04:09:22.656723+0000","last_undegraded":"2024-03-29T04:09:22.656723+0000","last_fullsized":"2024-03-29T04:09:22.656723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329391+0000","last_change":"2024-03-29T04:09:20.882224+0000","last_active":"2024-03-29T04:09:22.329391+0000","last_peered":"2024-03-29T04:09:22.329391+0000","last_clean":"2024-03-29T04:09:22.329391+0000","last_became_active":"2024-03-29T04:09:20.881813+0000","last_became_peered":"2024-03-29T04:09:20.881813+0000","last_unstale":"2024-03-29T04:09:22.329391+0000","last_undegraded":"2024-03-29T04:09:22.329391+0000","last_fullsized":"2024-03-29T04:09:22.329391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655284+0000","last_change":"2024-03-29T04:09:16.322983+0000","last_active":"2024-03-29T04:09:22.655284+0000","last_peered":"2024-03-29T04:09:22.655284+0000","last_clean":"2024-03-29T04:09:22.655284+0000","last_became_active":"2024-03-29T04:09:16.322712+0000","last_became_peered":"2024-03-29T04:09:16.322712+0000","last_unstale":"2024-03-29T04:09:22.655284+0000","last_undegraded":"2024-03-29T04:09:22.655284+0000","last_fullsized":"2024-03-29T04:09:22.655284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:15.302835+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:15.302365+0000","last_became_peered":"2024-03-29T04:09:15.302365+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655981+0000","last_change":"2024-03-29T04:09:20.347857+0000","last_active":"2024-03-29T04:09:22.655981+0000","last_peered":"2024-03-29T04:09:22.655981+0000","last_clean":"2024-03-29T04:09:22.655981+0000","last_became_active":"2024-03-29T04:09:20.347499+0000","last_became_peered":"2024-03-29T04:09:20.347499+0000","last_unstale":"2024-03-29T04:09:22.655981+0000","last_undegraded":"2024-03-29T04:09:22.655981+0000","last_fullsized":"2024-03-29T04:09:22.655981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656262+0000","last_change":"2024-03-29T04:09:16.321770+0000","last_active":"2024-03-29T04:09:22.656262+0000","last_peered":"2024-03-29T04:09:22.656262+0000","last_clean":"2024-03-29T04:09:22.656262+0000","last_became_active":"2024-03-29T04:09:16.321511+0000","last_became_peered":"2024-03-29T04:09:16.321511+0000","last_unstale":"2024-03-29T04:09:22.656262+0000","last_undegraded":"2024-03-29T04:09:22.656262+0000","last_fullsized":"2024-03-29T04:09:22.656262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656031+0000","last_change":"2024-03-29T04:09:15.306618+0000","last_active":"2024-03-29T04:09:22.656031+0000","last_peered":"2024-03-29T04:09:22.656031+0000","last_clean":"2024-03-29T04:09:22.656031+0000","last_became_active":"2024-03-29T04:09:15.306337+0000","last_became_peered":"2024-03-29T04:09:15.306337+0000","last_unstale":"2024-03-29T04:09:22.656031+0000","last_undegraded":"2024-03-29T04:09:22.656031+0000","last_fullsized":"2024-03-29T04:09:22.656031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654617+0000","last_change":"2024-03-29T04:09:20.884325+0000","last_active":"2024-03-29T04:09:22.654617+0000","last_peered":"2024-03-29T04:09:22.654617+0000","last_clean":"2024-03-29T04:09:22.654617+0000","last_became_active":"2024-03-29T04:09:20.883509+0000","last_became_peered":"2024-03-29T04:09:20.883509+0000","last_unstale":"2024-03-29T04:09:22.654617+0000","last_undegraded":"2024-03-29T04:09:22.654617+0000","last_fullsized":"2024-03-29T04:09:22.654617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:45:02.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655965+0000","last_change":"2024-03-29T04:09:16.309527+0000","last_active":"2024-03-29T04:09:22.655965+0000","last_peered":"2024-03-29T04:09:22.655965+0000","last_clean":"2024-03-29T04:09:22.655965+0000","last_became_active":"2024-03-29T04:09:16.309219+0000","last_became_peered":"2024-03-29T04:09:16.309219+0000","last_unstale":"2024-03-29T04:09:22.655965+0000","last_undegraded":"2024-03-29T04:09:22.655965+0000","last_fullsized":"2024-03-29T04:09:22.655965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329782+0000","last_change":"2024-03-29T04:09:15.306652+0000","last_active":"2024-03-29T04:09:22.329782+0000","last_peered":"2024-03-29T04:09:22.329782+0000","last_clean":"2024-03-29T04:09:22.329782+0000","last_became_active":"2024-03-29T04:09:15.306354+0000","last_became_peered":"2024-03-29T04:09:15.306354+0000","last_unstale":"2024-03-29T04:09:22.329782+0000","last_undegraded":"2024-03-29T04:09:22.329782+0000","last_fullsized":"2024-03-29T04:09:22.329782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330540+0000","last_change":"2024-03-29T04:09:20.346199+0000","last_active":"2024-03-29T04:09:22.330540+0000","last_peered":"2024-03-29T04:09:22.330540+0000","last_clean":"2024-03-29T04:09:22.330540+0000","last_became_active":"2024-03-29T04:09:20.345826+0000","last_became_peered":"2024-03-29T04:09:20.345826+0000","last_unstale":"2024-03-29T04:09:22.330540+0000","last_undegraded":"2024-03-29T04:09:22.330540+0000","last_fullsized":"2024-03-29T04:09:22.330540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.321785+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.321533+0000","last_became_peered":"2024-03-29T04:09:16.321533+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655385+0000","last_change":"2024-03-29T04:09:15.307664+0000","last_active":"2024-03-29T04:09:22.655385+0000","last_peered":"2024-03-29T04:09:22.655385+0000","last_clean":"2024-03-29T04:09:22.655385+0000","last_became_active":"2024-03-29T04:09:15.307412+0000","last_became_peered":"2024-03-29T04:09:15.307412+0000","last_unstale":"2024-03-29T04:09:22.655385+0000","last_undegraded":"2024-03-29T04:09:22.655385+0000","last_fullsized":"2024-03-29T04:09:22.655385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655742+0000","last_change":"2024-03-29T04:09:20.346476+0000","last_active":"2024-03-29T04:09:22.655742+0000","last_peered":"2024-03-29T04:09:22.655742+0000","last_clean":"2024-03-29T04:09:22.655742+0000","last_became_active":"2024-03-29T04:09:20.346140+0000","last_became_peered":"2024-03-29T04:09:20.346140+0000","last_unstale":"2024-03-29T04:09:22.655742+0000","last_undegraded":"2024-03-29T04:09:22.655742+0000","last_fullsized":"2024-03-29T04:09:22.655742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329068+0000","last_change":"2024-03-29T04:09:16.313068+0000","last_active":"2024-03-29T04:09:22.329068+0000","last_peered":"2024-03-29T04:09:22.329068+0000","last_clean":"2024-03-29T04:09:22.329068+0000","last_became_active":"2024-03-29T04:09:16.312754+0000","last_became_peered":"2024-03-29T04:09:16.312754+0000","last_unstale":"2024-03-29T04:09:22.329068+0000","last_undegraded":"2024-03-29T04:09:22.329068+0000","last_fullsized":"2024-03-29T04:09:22.329068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329238+0000","last_change":"2024-03-29T04:09:15.300287+0000","last_active":"2024-03-29T04:09:22.329238+0000","last_peered":"2024-03-29T04:09:22.329238+0000","last_clean":"2024-03-29T04:09:22.329238+0000","last_became_active":"2024-03-29T04:09:15.299961+0000","last_became_peered":"2024-03-29T04:09:15.299961+0000","last_unstale":"2024-03-29T04:09:22.329238+0000","last_undegraded":"2024-03-29T04:09:22.329238+0000","last_fullsized":"2024-03-29T04:09:22.329238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329627+0000","last_change":"2024-03-29T04:09:20.883520+0000","last_active":"2024-03-29T04:09:22.329627+0000","last_peered":"2024-03-29T04:09:22.329627+0000","last_clean":"2024-03-29T04:09:22.329627+0000","last_became_active":"2024-03-29T04:09:20.883152+0000","last_became_peered":"2024-03-29T04:09:20.883152+0000","last_unstale":"2024-03-29T04:09:22.329627+0000","last_undegraded":"2024-03-29T04:09:22.329627+0000","last_fullsized":"2024-03-29T04:09:22.329627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:25.394492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655644+0000","last_change":"2024-03-29T04:09:16.322576+0000","last_active":"2024-03-29T04:09:22.655644+0000","last_peered":"2024-03-29T04:09:22.655644+0000","last_clean":"2024-03-29T04:09:22.655644+0000","last_became_active":"2024-03-29T04:09:16.322126+0000","last_became_peered":"2024-03-29T04:09:16.322126+0000","last_unstale":"2024-03-29T04:09:22.655644+0000","last_undegraded":"2024-03-29T04:09:22.655644+0000","last_fullsized":"2024-03-29T04:09:22.655644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655221+0000","last_change":"2024-03-29T04:09:15.306114+0000","last_active":"2024-03-29T04:09:22.655221+0000","last_peered":"2024-03-29T04:09:22.655221+0000","last_clean":"2024-03-29T04:09:22.655221+0000","last_became_active":"2024-03-29T04:09:15.305893+0000","last_became_peered":"2024-03-29T04:09:15.305893+0000","last_unstale":"2024-03-29T04:09:22.655221+0000","last_undegraded":"2024-03-29T04:09:22.655221+0000","last_fullsized":"2024-03-29T04:09:22.655221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329246+0000","last_change":"2024-03-29T04:09:20.882277+0000","last_active":"2024-03-29T04:09:22.329246+0000","last_peered":"2024-03-29T04:09:22.329246+0000","last_clean":"2024-03-29T04:09:22.329246+0000","last_became_active":"2024-03-29T04:09:20.881878+0000","last_became_peered":"2024-03-29T04:09:20.881878+0000","last_unstale":"2024-03-29T04:09:22.329246+0000","last_undegraded":"2024-03-29T04:09:22.329246+0000","last_fullsized":"2024-03-29T04:09:22.329246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:06.012938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655785+0000","last_change":"2024-03-29T04:09:16.307882+0000","last_active":"2024-03-29T04:09:22.655785+0000","last_peered":"2024-03-29T04:09:22.655785+0000","last_clean":"2024-03-29T04:09:22.655785+0000","last_became_active":"2024-03-29T04:09:16.307620+0000","last_became_peered":"2024-03-29T04:09:16.307620+0000","last_unstale":"2024-03-29T04:09:22.655785+0000","last_undegraded":"2024-03-29T04:09:22.655785+0000","last_fullsized":"2024-03-29T04:09:22.655785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656164+0000","last_change":"2024-03-29T04:09:15.300985+0000","last_active":"2024-03-29T04:09:22.656164+0000","last_peered":"2024-03-29T04:09:22.656164+0000","last_clean":"2024-03-29T04:09:22.656164+0000","last_became_active":"2024-03-29T04:09:15.300603+0000","last_became_peered":"2024-03-29T04:09:15.300603+0000","last_unstale":"2024-03-29T04:09:22.656164+0000","last_undegraded":"2024-03-29T04:09:22.656164+0000","last_fullsized":"2024-03-29T04:09:22.656164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:12.516429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655363+0000","last_change":"2024-03-29T04:09:20.883953+0000","last_active":"2024-03-29T04:09:22.655363+0000","last_peered":"2024-03-29T04:09:22.655363+0000","last_clean":"2024-03-29T04:09:22.655363+0000","last_became_active":"2024-03-29T04:09:20.883597+0000","last_became_peered":"2024-03-29T04:09:20.883597+0000","last_unstale":"2024-03-29T04:09:22.655363+0000","last_undegraded":"2024-03-29T04:09:22.655363+0000","last_fullsized":"2024-03-29T04:09:22.655363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:27.295430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655864+0000","last_change":"2024-03-29T04:09:16.306905+0000","last_active":"2024-03-29T04:09:22.655864+0000","last_peered":"2024-03-29T04:09:22.655864+0000","last_clean":"2024-03-29T04:09:22.655864+0000","last_became_active":"2024-03-29T04:09:16.306156+0000","last_became_peered":"2024-03-29T04:09:16.306156+0000","last_unstale":"2024-03-29T04:09:22.655864+0000","last_undegraded":"2024-03-29T04:09:22.655864+0000","last_fullsized":"2024-03-29T04:09:22.655864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:25.361360+0000","last_change":"2024-03-29T04:09:15.302359+0000","last_active":"2024-03-29T04:09:25.361360+0000","last_peered":"2024-03-29T04:09:25.361360+0000","last_clean":"2024-03-29T04:09:25.361360+0000","last_became_active":"2024-03-29T04:09:15.302044+0000","last_became_peered":"2024-03-29T04:09:15.302044+0000","last_unstale":"2024-03-29T04:09:25.361360+0000","last_undegraded":"2024-03-29T04:09:25.361360+0000","last_fullsized":"2024-03-29T04:09:25.361360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656202+0000","last_change":"2024-03-29T04:09:20.352101+0000","last_active":"2024-03-29T04:09:22.656202+0000","last_peered":"2024-03-29T04:09:22.656202+0000","last_clean":"2024-03-29T04:09:22.656202+0000","last_became_active":"2024-03-29T04:09:20.351597+0000","last_became_peered":"2024-03-29T04:09:20.351597+0000","last_unstale":"2024-03-29T04:09:22.656202+0000","last_undegraded":"2024-03-29T04:09:22.656202+0000","last_fullsized":"2024-03-29T04:09:22.656202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330245+0000","last_change":"2024-03-29T04:09:16.305800+0000","last_active":"2024-03-29T04:09:22.330245+0000","last_peered":"2024-03-29T04:09:22.330245+0000","last_clean":"2024-03-29T04:09:22.330245+0000","last_became_active":"2024-03-29T04:09:16.305534+0000","last_became_peered":"2024-03-29T04:09:16.305534+0000","last_unstale":"2024-03-29T04:09:22.330245+0000","last_undegraded":"2024-03-29T04:09:22.330245+0000","last_fullsized":"2024-03-29T04:09:22.330245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655674+0000","last_change":"2024-03-29T04:09:15.309193+0000","last_active":"2024-03-29T04:09:22.655674+0000","last_peered":"2024-03-29T04:09:22.655674+0000","last_clean":"2024-03-29T04:09:22.655674+0000","last_became_active":"2024-03-29T04:09:15.308884+0000","last_became_peered":"2024-03-29T04:09:15.308884+0000","last_unstale":"2024-03-29T04:09:22.655674+0000","last_undegraded":"2024-03-29T04:09:22.655674+0000","last_fullsized":"2024-03-29T04:09:22.655674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330135+0000","last_change":"2024-03-29T04:09:20.348029+0000","last_active":"2024-03-29T04:09:22.330135+0000","last_peered":"2024-03-29T04:09:22.330135+0000","last_clean":"2024-03-29T04:09:22.330135+0000","last_became_active":"2024-03-29T04:09:20.347196+0000","last_became_peered":"2024-03-29T04:09:20.347196+0000","last_unstale":"2024-03-29T04:09:22.330135+0000","last_undegraded":"2024-03-29T04:09:22.330135+0000","last_fullsized":"2024-03-29T04:09:22.330135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656194+0000","last_change":"2024-03-29T04:09:16.318939+0000","last_active":"2024-03-29T04:09:22.656194+0000","last_peered":"2024-03-29T04:09:22.656194+0000","last_clean":"2024-03-29T04:09:22.656194+0000","last_became_active":"2024-03-29T04:09:16.318327+0000","last_became_peered":"2024-03-29T04:09:16.318327+0000","last_unstale":"2024-03-29T04:09:22.656194+0000","last_undegraded":"2024-03-29T04:09:22.656194+0000","last_fullsized":"2024-03-29T04:09:22.656194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654888+0000","last_change":"2024-03-29T04:09:15.300878+0000","last_active":"2024-03-29T04:09:22.654888+0000","last_peered":"2024-03-29T04:09:22.654888+0000","last_clean":"2024-03-29T04:09:22.654888+0000","last_became_active":"2024-03-29T04:09:15.300567+0000","last_became_peered":"2024-03-29T04:09:15.300567+0000","last_unstale":"2024-03-29T04:09:22.654888+0000","last_undegraded":"2024-03-29T04:09:22.654888+0000","last_fullsized":"2024-03-29T04:09:22.654888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328959+0000","last_change":"2024-03-29T04:09:20.343510+0000","last_active":"2024-03-29T04:09:22.328959+0000","last_peered":"2024-03-29T04:09:22.328959+0000","last_clean":"2024-03-29T04:09:22.328959+0000","last_became_active":"2024-03-29T04:09:20.343070+0000","last_became_peered":"2024-03-29T04:09:20.343070+0000","last_unstale":"2024-03-29T04:09:22.328959+0000","last_undegraded":"2024-03-29T04:09:22.328959+0000","last_fullsized":"2024-03-29T04:09:22.328959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:10.010570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654868+0000","last_change":"2024-03-29T04:09:16.314479+0000","last_active":"2024-03-29T04:09:22.654868+0000","last_peered":"2024-03-29T04:09:22.654868+0000","last_clean":"2024-03-29T04:09:22.654868+0000","last_became_active":"2024-03-29T04:09:16.313990+0000","last_became_peered":"2024-03-29T04:09:16.313990+0000","last_unstale":"2024-03-29T04:09:22.654868+0000","last_undegraded":"2024-03-29T04:09:22.654868+0000","last_fullsized":"2024-03-29T04:09:22.654868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329948+0000","last_change":"2024-03-29T04:09:15.304665+0000","last_active":"2024-03-29T04:09:22.329948+0000","last_peered":"2024-03-29T04:09:22.329948+0000","last_clean":"2024-03-29T04:09:22.329948+0000","last_became_active":"2024-03-29T04:09:15.304381+0000","last_became_peered":"2024-03-29T04:09:15.304381+0000","last_unstale":"2024-03-29T04:09:22.329948+0000","last_undegraded":"2024-03-29T04:09:22.329948+0000","last_fullsized":"2024-03-29T04:09:22.329948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329035+0000","last_change":"2024-03-29T04:09:20.884477+0000","last_active":"2024-03-29T04:09:22.329035+0000","last_peered":"2024-03-29T04:09:22.329035+0000","last_clean":"2024-03-29T04:09:22.329035+0000","last_became_active":"2024-03-29T04:09:20.884123+0000","last_became_peered":"2024-03-29T04:09:20.884123+0000","last_unstale":"2024-03-29T04:09:22.329035+0000","last_undegraded":"2024-03-29T04:09:22.329035+0000","last_fullsized":"2024-03-29T04:09:22.329035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:11.194987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655599+0000","last_change":"2024-03-29T04:09:20.883017+0000","last_active":"2024-03-29T04:09:22.655599+0000","last_peered":"2024-03-29T04:09:22.655599+0000","last_clean":"2024-03-29T04:09:22.655599+0000","last_became_active":"2024-03-29T04:09:20.882655+0000","last_became_peered":"2024-03-29T04:09:20.882655+0000","last_unstale":"2024-03-29T04:09:22.655599+0000","last_undegraded":"2024-03-29T04:09:22.655599+0000","last_fullsized":"2024-03-29T04:09:22.655599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655207+0000","last_change":"2024-03-29T04:09:16.313418+0000","last_active":"2024-03-29T04:09:22.655207+0000","last_peered":"2024-03-29T04:09:22.655207+0000","last_clean":"2024-03-29T04:09:22.655207+0000","last_became_active":"2024-03-29T04:09:16.312923+0000","last_became_peered":"2024-03-29T04:09:16.312923+0000","last_unstale":"2024-03-29T04:09:22.655207+0000","last_undegraded":"2024-03-29T04:09:22.655207+0000","last_fullsized":"2024-03-29T04:09:22.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654732+0000","last_change":"2024-03-29T04:09:15.304449+0000","last_active":"2024-03-29T04:09:22.654732+0000","last_peered":"2024-03-29T04:09:22.654732+0000","last_clean":"2024-03-29T04:09:22.654732+0000","last_became_active":"2024-03-29T04:09:15.304164+0000","last_became_peered":"2024-03-29T04:09:15.304164+0000","last_unstale":"2024-03-29T04:09:22.654732+0000","last_undegraded":"2024-03-29T04:09:22.654732+0000","last_fullsized":"2024-03-29T04:09:22.654732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329921+0000","last_change":"2024-03-29T04:09:20.344669+0000","last_active":"2024-03-29T04:09:22.329921+0000","last_peered":"2024-03-29T04:09:22.329921+0000","last_clean":"2024-03-29T04:09:22.329921+0000","last_became_active":"2024-03-29T04:09:20.344359+0000","last_became_peered":"2024-03-29T04:09:20.344359+0000","last_unstale":"2024-03-29T04:09:22.329921+0000","last_undegraded":"2024-03-29T04:09:22.329921+0000","last_fullsized":"2024-03-29T04:09:22.329921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328865+0000","last_change":"2024-03-29T04:09:16.308421+0000","last_active":"2024-03-29T04:09:22.328865+0000","last_peered":"2024-03-29T04:09:22.328865+0000","last_clean":"2024-03-29T04:09:22.328865+0000","last_became_active":"2024-03-29T04:09:16.308142+0000","last_became_peered":"2024-03-29T04:09:16.308142+0000","last_unstale":"2024-03-29T04:09:22.328865+0000","last_undegraded":"2024-03-29T04:09:22.328865+0000","last_fullsized":"2024-03-29T04:09:22.328865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329237+0000","last_change":"2024-03-29T04:09:15.302691+0000","last_active":"2024-03-29T04:09:22.329237+0000","last_peered":"2024-03-29T04:09:22.329237+0000","last_clean":"2024-03-29T04:09:22.329237+0000","last_became_active":"2024-03-29T04:09:15.302305+0000","last_became_peered":"2024-03-29T04:09:15.302305+0000","last_unstale":"2024-03-29T04:09:22.329237+0000","last_undegraded":"2024-03-29T04:09:22.329237+0000","last_fullsized":"2024-03-29T04:09:22.329237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329606+0000","last_change":"2024-03-29T04:09:15.299655+0000","last_active":"2024-03-29T04:09:22.329606+0000","last_peered":"2024-03-29T04:09:22.329606+0000","last_clean":"2024-03-29T04:09:22.329606+0000","last_became_active":"2024-03-29T04:09:15.299361+0000","last_became_peered":"2024-03-29T04:09:15.299361+0000","last_unstale":"2024-03-29T04:09:22.329606+0000","last_undegraded":"2024-03-29T04:09:22.329606+0000","last_fullsized":"2024-03-29T04:09:22.329606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656124+0000","last_change":"2024-03-29T04:09:16.322096+0000","last_active":"2024-03-29T04:09:22.656124+0000","last_peered":"2024-03-29T04:09:22.656124+0000","last_clean":"2024-03-29T04:09:22.656124+0000","last_became_active":"2024-03-29T04:09:16.321833+0000","last_became_peered":"2024-03-29T04:09:16.321833+0000","last_unstale":"2024-03-29T04:09:22.656124+0000","last_undegraded":"2024-03-29T04:09:22.656124+0000","last_fullsized":"2024-03-29T04:09:22.656124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656607+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.656607+0000","last_peered":"2024-03-29T04:09:22.656607+0000","last_clean":"2024-03-29T04:09:22.656607+0000","last_became_active":"2024-03-29T04:09:20.347088+0000","last_became_peered":"2024-03-29T04:09:20.347088+0000","last_unstale":"2024-03-29T04:09:22.656607+0000","last_undegraded":"2024-03-29T04:09:22.656607+0000","last_fullsized":"2024-03-29T04:09:22.656607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:45.089535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329185+0000","last_change":"2024-03-29T04:09:15.301823+0000","last_active":"2024-03-29T04:09:22.329185+0000","last_peered":"2024-03-29T04:09:22.329185+0000","last_clean":"2024-03-29T04:09:22.329185+0000","last_became_active":"2024-03-29T04:09:15.301182+0000","last_became_peered":"2024-03-29T04:09:15.301182+0000","last_unstale":"2024-03-29T04:09:22.329185+0000","last_undegraded":"2024-03-29T04:09:22.329185+0000","last_fullsized":"2024-03-29T04:09:22.329185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329886+0000","last_change":"2024-03-29T04:09:16.311797+0000","last_active":"2024-03-29T04:09:22.329886+0000","last_peered":"2024-03-29T04:09:22.329886+0000","last_clean":"2024-03-29T04:09:22.329886+0000","last_became_active":"2024-03-29T04:09:16.311544+0000","last_became_peered":"2024-03-29T04:09:16.311544+0000","last_unstale":"2024-03-29T04:09:22.329886+0000","last_undegraded":"2024-03-29T04:09:22.329886+0000","last_fullsized":"2024-03-29T04:09:22.329886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:41.184352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329592+0000","last_change":"2024-03-29T04:09:20.344647+0000","last_active":"2024-03-29T04:09:22.329592+0000","last_peered":"2024-03-29T04:09:22.329592+0000","last_clean":"2024-03-29T04:09:22.329592+0000","last_became_active":"2024-03-29T04:09:20.344278+0000","last_became_peered":"2024-03-29T04:09:20.344278+0000","last_unstale":"2024-03-29T04:09:22.329592+0000","last_undegraded":"2024-03-29T04:09:22.329592+0000","last_fullsized":"2024-03-29T04:09:22.329592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:15.627313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328975+0000","last_change":"2024-03-29T04:09:16.311872+0000","last_active":"2024-03-29T04:09:22.328975+0000","last_peered":"2024-03-29T04:09:22.328975+0000","last_clean":"2024-03-29T04:09:22.328975+0000","last_became_active":"2024-03-29T04:09:16.311576+0000","last_became_peered":"2024-03-29T04:09:16.311576+0000","last_unstale":"2024-03-29T04:09:22.328975+0000","last_undegraded":"2024-03-29T04:09:22.328975+0000","last_fullsized":"2024-03-29T04:09:22.328975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656255+0000","last_change":"2024-03-29T04:09:16.315268+0000","last_active":"2024-03-29T04:09:22.656255+0000","last_peered":"2024-03-29T04:09:22.656255+0000","last_clean":"2024-03-29T04:09:22.656255+0000","last_became_active":"2024-03-29T04:09:16.315002+0000","last_became_peered":"2024-03-29T04:09:16.315002+0000","last_unstale":"2024-03-29T04:09:22.656255+0000","last_undegraded":"2024-03-29T04:09:22.656255+0000","last_fullsized":"2024-03-29T04:09:22.656255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.323782+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.323504+0000","last_became_peered":"2024-03-29T04:09:16.323504+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656037+0000","last_change":"2024-03-29T04:09:16.316947+0000","last_active":"2024-03-29T04:09:22.656037+0000","last_peered":"2024-03-29T04:09:22.656037+0000","last_clean":"2024-03-29T04:09:22.656037+0000","last_became_active":"2024-03-29T04:09:16.316689+0000","last_became_peered":"2024-03-29T04:09:16.316689+0000","last_unstale":"2024-03-29T04:09:22.656037+0000","last_undegraded":"2024-03-29T04:09:22.656037+0000","last_fullsized":"2024-03-29T04:09:22.656037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329288+0000","last_change":"2024-03-29T04:09:16.319962+0000","last_active":"2024-03-29T04:09:22.329288+0000","last_peered":"2024-03-29T04:09:22.329288+0000","last_clean":"2024-03-29T04:09:22.329288+0000","last_became_active":"2024-03-29T04:09:16.319316+0000","last_became_peered":"2024-03-29T04:09:16.319316+0000","last_unstale":"2024-03-29T04:09:22.329288+0000","last_undegraded":"2024-03-29T04:09:22.329288+0000","last_fullsized":"2024-03-29T04:09:22.329288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655509+0000","last_change":"2024-03-29T04:09:16.312227+0000","last_active":"2024-03-29T04:09:22.655509+0000","last_peered":"2024-03-29T04:09:22.655509+0000","last_clean":"2024-03-29T04:09:22.655509+0000","last_became_active":"2024-03-29T04:09:16.311944+0000","last_became_peered":"2024-03-29T04:09:16.311944+0000","last_unstale":"2024-03-29T04:09:22.655509+0000","last_undegraded":"2024-03-29T04:09:22.655509+0000","last_fullsized":"2024-03-29T04:09:22.655509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330212+0000","last_change":"2024-03-29T04:09:16.306234+0000","last_active":"2024-03-29T04:09:22.330212+0000","last_peered":"2024-03-29T04:09:22.330212+0000","last_clean":"2024-03-29T04:09:22.330212+0000","last_became_active":"2024-03-29T04:09:16.305931+0000","last_became_peered":"2024-03-29T04:09:16.305931+0000","last_unstale":"2024-03-29T04:09:22.330212+0000","last_undegraded":"2024-03-29T04:09:22.330212+0000","last_fullsized":"2024-03-29T04:09:22.330212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329471+0000","last_change":"2024-03-29T04:09:16.310048+0000","last_active":"2024-03-29T04:09:22.329471+0000","last_peered":"2024-03-29T04:09:22.329471+0000","last_clean":"2024-03-29T04:09:22.329471+0000","last_became_active":"2024-03-29T04:09:16.309626+0000","last_became_peered":"2024-03-29T04:09:16.309626+0000","last_unstale":"2024-03-29T04:09:22.329471+0000","last_undegraded":"2024-03-29T04:09:22.329471+0000","last_fullsized":"2024-03-29T04:09:22.329471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654709+0000","last_change":"2024-03-29T04:09:16.324820+0000","last_active":"2024-03-29T04:09:22.654709+0000","last_peered":"2024-03-29T04:09:22.654709+0000","last_clean":"2024-03-29T04:09:22.654709+0000","last_became_active":"2024-03-29T04:09:16.324516+0000","last_became_peered":"2024-03-29T04:09:16.324516+0000","last_unstale":"2024-03-29T04:09:22.654709+0000","last_undegraded":"2024-03-29T04:09:22.654709+0000","last_fullsized":"2024-03-29T04:09:22.654709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329113+0000","last_change":"2024-03-29T04:09:16.318107+0000","last_active":"2024-03-29T04:09:22.329113+0000","last_peered":"2024-03-29T04:09:22.329113+0000","last_clean":"2024-03-29T04:09:22.329113+0000","last_became_active":"2024-03-29T04:09:16.317633+0000","last_became_peered":"2024-03-29T04:09:16.317633+0000","last_unstale":"2024-03-29T04:09:22.329113+0000","last_undegraded":"2024-03-29T04:09:22.329113+0000","last_fullsized":"2024-03-29T04:09:22.329113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654968+0000","last_change":"2024-03-29T04:09:16.317756+0000","last_active":"2024-03-29T04:09:22.654968+0000","last_peered":"2024-03-29T04:09:22.654968+0000","last_clean":"2024-03-29T04:09:22.654968+0000","last_became_active":"2024-03-29T04:09:16.317411+0000","last_became_peered":"2024-03-29T04:09:16.317411+0000","last_unstale":"2024-03-29T04:09:22.654968+0000","last_undegraded":"2024-03-29T04:09:22.654968+0000","last_fullsized":"2024-03-29T04:09:22.654968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655375+0000","last_change":"2024-03-29T04:09:16.314033+0000","last_active":"2024-03-29T04:09:22.655375+0000","last_peered":"2024-03-29T04:09:22.655375+0000","last_clean":"2024-03-29T04:09:22.655375+0000","last_became_active":"2024-03-29T04:09:16.313440+0000","last_became_peered":"2024-03-29T04:09:16.313440+0000","last_unstale":"2024-03-29T04:09:22.655375+0000","last_undegraded":"2024-03-29T04:09:22.655375+0000","last_fullsized":"2024-03-29T04:09:22.655375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329544+0000","last_change":"2024-03-29T04:09:16.318843+0000","last_active":"2024-03-29T04:09:22.329544+0000","last_peered":"2024-03-29T04:09:22.329544+0000","last_clean":"2024-03-29T04:09:22.329544+0000","last_became_active":"2024-03-29T04:09:16.318563+0000","last_became_peered":"2024-03-29T04:09:16.318563+0000","last_unstale":"2024-03-29T04:09:22.329544+0000","last_undegraded":"2024-03-29T04:09:22.329544+0000","last_fullsized":"2024-03-29T04:09:22.329544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656080+0000","last_change":"2024-03-29T04:09:16.313733+0000","last_active":"2024-03-29T04:09:22.656080+0000","last_peered":"2024-03-29T04:09:22.656080+0000","last_clean":"2024-03-29T04:09:22.656080+0000","last_became_active":"2024-03-29T04:09:16.313193+0000","last_became_peered":"2024-03-29T04:09:16.313193+0000","last_unstale":"2024-03-29T04:09:22.656080+0000","last_undegraded":"2024-03-29T04:09:22.656080+0000","last_fullsized":"2024-03-29T04:09:22.656080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:59.588666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655943+0000","last_change":"2024-03-29T04:09:16.300679+0000","last_active":"2024-03-29T04:09:22.655943+0000","last_peered":"2024-03-29T04:09:22.655943+0000","last_clean":"2024-03-29T04:09:22.655943+0000","last_became_active":"2024-03-29T04:09:16.300399+0000","last_became_peered":"2024-03-29T04:09:16.300399+0000","last_unstale":"2024-03-29T04:09:22.655943+0000","last_undegraded":"2024-03-29T04:09:22.655943+0000","last_fullsized":"2024-03-29T04:09:22.655943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:10.179988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655462+0000","last_change":"2024-03-29T04:09:16.314857+0000","last_active":"2024-03-29T04:09:22.655462+0000","last_peered":"2024-03-29T04:09:22.655462+0000","last_clean":"2024-03-29T04:09:22.655462+0000","last_became_active":"2024-03-29T04:09:16.314478+0000","last_became_peered":"2024-03-29T04:09:16.314478+0000","last_unstale":"2024-03-29T04:09:22.655462+0000","last_undegraded":"2024-03-29T04:09:22.655462+0000","last_fullsized":"2024-03-29T04:09:22.655462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654626+0000","last_change":"2024-03-29T04:09:16.314396+0000","last_active":"2024-03-29T04:09:22.654626+0000","last_peered":"2024-03-29T04:09:22.654626+0000","last_clean":"2024-03-29T04:09:22.654626+0000","last_became_active":"2024-03-29T04:09:16.313977+0000","last_became_peered":"2024-03-29T04:09:16.313977+0000","last_unstale":"2024-03-29T04:09:22.654626+0000","last_undegraded":"2024-03-29T04:09:22.654626+0000","last_fullsized":"2024-03-29T04:09:22.654626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655312+0000","last_change":"2024-03-29T04:09:16.320206+0000","last_active":"2024-03-29T04:09:22.655312+0000","last_peered":"2024-03-29T04:09:22.655312+0000","last_clean":"2024-03-29T04:09:22.655312+0000","last_became_active":"2024-03-29T04:09:16.319909+0000","last_became_peered":"2024-03-29T04:09:16.319909+0000","last_unstale":"2024-03-29T04:09:22.655312+0000","last_undegraded":"2024-03-29T04:09:22.655312+0000","last_fullsized":"2024-03-29T04:09:22.655312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:16.308762+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:16.308446+0000","last_became_peered":"2024-03-29T04:09:16.308446+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:27.406089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328992+0000","last_change":"2024-03-29T04:09:16.306843+0000","last_active":"2024-03-29T04:09:22.328992+0000","last_peered":"2024-03-29T04:09:22.328992+0000","last_clean":"2024-03-29T04:09:22.328992+0000","last_became_active":"2024-03-29T04:09:16.306402+0000","last_became_peered":"2024-03-29T04:09:16.306402+0000","last_unstale":"2024-03-29T04:09:22.328992+0000","last_undegraded":"2024-03-29T04:09:22.328992+0000","last_fullsized":"2024-03-29T04:09:22.328992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330172+0000","last_change":"2024-03-29T04:09:16.310846+0000","last_active":"2024-03-29T04:09:22.330172+0000","last_peered":"2024-03-29T04:09:22.330172+0000","last_clean":"2024-03-29T04:09:22.330172+0000","last_became_active":"2024-03-29T04:09:16.309820+0000","last_became_peered":"2024-03-29T04:09:16.309820+0000","last_unstale":"2024-03-29T04:09:22.330172+0000","last_undegraded":"2024-03-29T04:09:22.330172+0000","last_fullsized":"2024-03-29T04:09:22.330172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:18.601851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329788+0000","last_change":"2024-03-29T04:09:16.316170+0000","last_active":"2024-03-29T04:09:22.329788+0000","last_peered":"2024-03-29T04:09:22.329788+0000","last_clean":"2024-03-29T04:09:22.329788+0000","last_became_active":"2024-03-29T04:09:16.315896+0000","last_became_peered":"2024-03-29T04:09:16.315896+0000","last_unstale":"2024-03-29T04:09:22.329788+0000","last_undegraded":"2024-03-29T04:09:22.329788+0000","last_fullsized":"2024-03-29T04:09:22.329788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330150+0000","last_change":"2024-03-29T04:09:16.316842+0000","last_active":"2024-03-29T04:09:22.330150+0000","last_peered":"2024-03-29T04:09:22.330150+0000","last_clean":"2024-03-29T04:09:22.330150+0000","last_became_active":"2024-03-29T04:09:16.316573+0000","last_became_peered":"2024-03-29T04:09:16.316573+0000","last_unstale":"2024-03-29T04:09:22.330150+0000","last_undegraded":"2024-03-29T04:09:22.330150+0000","last_fullsized":"2024-03-29T04:09:22.330150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656222+0000","last_change":"2024-03-29T04:09:16.316209+0000","last_active":"2024-03-29T04:09:22.656222+0000","last_peered":"2024-03-29T04:09:22.656222+0000","last_clean":"2024-03-29T04:09:22.656222+0000","last_became_active":"2024-03-29T04:09:16.315942+0000","last_became_peered":"2024-03-29T04:09:16.315942+0000","last_unstale":"2024-03-29T04:09:22.656222+0000","last_undegraded":"2024-03-29T04:09:22.656222+0000","last_fullsized":"2024-03-29T04:09:22.656222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328767+0000","last_change":"2024-03-29T04:09:16.316731+0000","last_active":"2024-03-29T04:09:22.328767+0000","last_peered":"2024-03-29T04:09:22.328767+0000","last_clean":"2024-03-29T04:09:22.328767+0000","last_became_active":"2024-03-29T04:09:16.316418+0000","last_became_peered":"2024-03-29T04:09:16.316418+0000","last_unstale":"2024-03-29T04:09:22.328767+0000","last_undegraded":"2024-03-29T04:09:22.328767+0000","last_fullsized":"2024-03-29T04:09:22.328767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.312822+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.312495+0000","last_became_peered":"2024-03-29T04:09:16.312495+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654882+0000","last_change":"2024-03-29T04:09:16.316248+0000","last_active":"2024-03-29T04:09:22.654882+0000","last_peered":"2024-03-29T04:09:22.654882+0000","last_clean":"2024-03-29T04:09:22.654882+0000","last_became_active":"2024-03-29T04:09:16.315901+0000","last_became_peered":"2024-03-29T04:09:16.315901+0000","last_unstale":"2024-03-29T04:09:22.654882+0000","last_undegraded":"2024-03-29T04:09:22.654882+0000","last_fullsized":"2024-03-29T04:09:22.654882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655574+0000","last_change":"2024-03-29T04:09:16.307754+0000","last_active":"2024-03-29T04:09:22.655574+0000","last_peered":"2024-03-29T04:09:22.655574+0000","last_clean":"2024-03-29T04:09:22.655574+0000","last_became_active":"2024-03-29T04:09:16.307322+0000","last_became_peered":"2024-03-29T04:09:16.307322+0000","last_unstale":"2024-03-29T04:09:22.655574+0000","last_undegraded":"2024-03-29T04:09:22.655574+0000","last_fullsized":"2024-03-29T04:09:22.655574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.323875+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.323578+0000","last_became_peered":"2024-03-29T04:09:16.323578+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655497+0000","last_change":"2024-03-29T04:09:16.311321+0000","last_active":"2024-03-29T04:09:22.655497+0000","last_peered":"2024-03-29T04:09:22.655497+0000","last_clean":"2024-03-29T04:09:22.655497+0000","last_became_active":"2024-03-29T04:09:16.310990+0000","last_became_peered":"2024-03-29T04:09:16.310990+0000","last_unstale":"2024-03-29T04:09:22.655497+0000","last_undegraded":"2024-03-29T04:09:22.655497+0000","last_fullsized":"2024-03-29T04:09:22.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656870+0000","last_change":"2024-03-29T04:09:16.313513+0000","last_active":"2024-03-29T04:09:22.656870+0000","last_peered":"2024-03-29T04:09:22.656870+0000","last_clean":"2024-03-29T04:09:22.656870+0000","last_became_active":"2024-03-29T04:09:16.312390+0000","last_became_peered":"2024-03-29T04:09:16.312390+0000","last_unstale":"2024-03-29T04:09:22.656870+0000","last_undegraded":"2024-03-29T04:09:22.656870+0000","last_fullsized":"2024-03-29T04:09:22.656870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656162+0000","last_change":"2024-03-29T04:09:16.303994+0000","last_active":"2024-03-29T04:09:22.656162+0000","last_peered":"2024-03-29T04:09:22.656162+0000","last_clean":"2024-03-29T04:09:22.656162+0000","last_became_active":"2024-03-29T04:09:16.303670+0000","last_became_peered":"2024-03-29T04:09:16.303670+0000","last_unstale":"2024-03-29T04:09:22.656162+0000","last_undegraded":"2024-03-29T04:09:22.656162+0000","last_fullsized":"2024-03-29T04:09:22.656162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:33.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.464+0000 7f6abdcf2640 1 Processor -- start 2024-03-29T04:09:33.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.464+0000 7f6abdcf2640 1 -- start start 2024-03-29T04:09:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6abdcf2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ab8141530 0x7f6ab8071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6abdcf2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ab8162030 0x7f6ab81624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6abdcf2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab8141910 con 0x7f6ab80b5240 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6abdcf2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab8071e60 con 0x7f6ab8141530 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6abdcf2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab8162af0 con 0x7f6ab8162030 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6ab77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ab8141530 0x7f6ab8071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6ab77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ab8141530 0x7f6ab8071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34354/0 (socket says 172.21.15.181:34354) 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6ab77fe640 1 -- 172.21.15.181:0/3386707399 learned_addr learned my addr 172.21.15.181:0/3386707399 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.465+0000 7f6aaeffd640 1 --2- 172.21.15.181:0/3386707399 >> v2:172.21.15.187:3300/0 conn(0x7f6ab8162030 0x7f6ab81624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.466+0000 7f6ab77fe640 1 -- 172.21.15.181:0/3386707399 >> v2:172.21.15.187:3300/0 conn(0x7f6ab8162030 msgr2=0x7f6ab81624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.466+0000 7f6ab77fe640 1 --2- 172.21.15.181:0/3386707399 >> v2:172.21.15.187:3300/0 conn(0x7f6ab8162030 0x7f6ab81624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.466+0000 7f6ab77fe640 1 -- 172.21.15.181:0/3386707399 >> v1:172.21.15.187:6789/0 conn(0x7f6ab80b5240 legacy=0x7f6ab8141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.466+0000 7f6ab77fe640 1 -- 172.21.15.181:0/3386707399 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab8162dd0 con 0x7f6ab8141530 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.466+0000 7f6ab77fe640 1 --2- 172.21.15.181:0/3386707399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8141530 0x7f6ab8071920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8161f70 tx=0x7f6aa802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60df030e1f08761 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6ab6ffd640 1 -- 172.21.15.181:0/3386707399 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8035020 con 0x7f6ab8141530 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6ab6ffd640 1 -- 172.21.15.181:0/3386707399 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa8031870 con 0x7f6ab8141530 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6ab6ffd640 1 -- 172.21.15.181:0/3386707399 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8031c40 con 0x7f6ab8141530 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 -- 172.21.15.181:0/3386707399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8141530 msgr2=0x7f6ab8071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/3386707399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8141530 0x7f6ab8071920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8161f70 tx=0x7f6aa802b910 comp rx=0 tx=0).stop 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 -- 172.21.15.181:0/3386707399 shutdown_connections 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/3386707399 >> v2:172.21.15.187:3300/0 conn(0x7f6ab8162030 0x7f6ab81624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/3386707399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8141530 0x7f6ab8071920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 -- 172.21.15.181:0/3386707399 >> 172.21.15.181:0/3386707399 conn(0x7f6ab806f8b0 msgr2=0x7f6ab806fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 -- 172.21.15.181:0/3386707399 shutdown_connections 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.467+0000 7f6abdcf2640 1 -- 172.21.15.181:0/3386707399 wait complete. 2024-03-29T04:09:33.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.468+0000 7f6abdcf2640 1 Processor -- start 2024-03-29T04:09:33.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 -- start start 2024-03-29T04:09:33.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab80b5240 0x7f6ab815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ab8141530 0x7f6ab81579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab80b1950 con 0x7f6ab8162030 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8071400 con 0x7f6ab80b5240 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6abdcf2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab8142320 con 0x7f6ab8141530 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6ab7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6ab7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34366/0 (socket says 172.21.15.181:34366) 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6ab7fff640 1 -- 172.21.15.181:0/2388991163 learned_addr learned my addr 172.21.15.181:0/2388991163 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6ab77fe640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab80b5240 0x7f6ab815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.469+0000 7f6aaeffd640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ab8141530 0x7f6ab81579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 -- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ab8141530 msgr2=0x7f6ab81579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ab8141530 0x7f6ab81579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 -- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab80b5240 msgr2=0x7f6ab815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab80b5240 0x7f6ab815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 -- 172.21.15.181:0/2388991163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab815e0b0 con 0x7f6ab8162030 2024-03-29T04:09:33.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab7fff640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c7a0 tx=0x7f6ab000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0003b70 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab0003d30 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab815cad0 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000aa90 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.470+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab815d090 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.471+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab8075a10 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.471+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ab000ac30 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.472+0000 7f6ab4ff9640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 0x7f6a9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.472+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6ab008b850 con 0x7f6ab8162030 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.472+0000 7f6ab77fe640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 0x7f6a9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.472+0000 7f6ab77fe640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 0x7f6a9806e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8036000 tx=0x7f6aa803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.476+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6ab00594e0 con 0x7f6ab8162030 2024-03-29T04:09:33.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.648+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ab80d2f00 con 0x7f6a9806c530 2024-03-29T04:09:33.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.655+0000 7f6ab4ff9640 1 -- 172.21.15.181:0/2388991163 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f6aa8099310 con 0x7f6a9806c530 2024-03-29T04:09:33.639 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:33.641 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:09:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 msgr2=0x7f6a9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 0x7f6a9806e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8036000 tx=0x7f6aa803c040 comp rx=0 tx=0).stop 2024-03-29T04:09:33.643 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 msgr2=0x7f6ab80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000c7a0 tx=0x7f6ab000cc60 comp rx=0 tx=0).stop 2024-03-29T04:09:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 shutdown_connections 2024-03-29T04:09:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab8162030 0x7f6ab80cb450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6a9806c530 0x7f6a9806e9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ab8141530 0x7f6ab81579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 --2- 172.21.15.181:0/2388991163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab80b5240 0x7f6ab815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 >> 172.21.15.181:0/2388991163 conn(0x7f6ab806f8b0 msgr2=0x7f6ab80b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 shutdown_connections 2024-03-29T04:09:33.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.659+0000 7f6abdcf2640 1 -- 172.21.15.181:0/2388991163 wait complete. 2024-03-29T04:09:33.650 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-03-29T04:09:33.577152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2048878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.249683"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656335+0000","last_change":"2024-03-29T04:09:15.308724+0000","last_active":"2024-03-29T04:09:22.656335+0000","last_peered":"2024-03-29T04:09:22.656335+0000","last_clean":"2024-03-29T04:09:22.656335+0000","last_became_active":"2024-03-29T04:09:15.308445+0000","last_became_peered":"2024-03-29T04:09:15.308445+0000","last_unstale":"2024-03-29T04:09:22.656335+0000","last_undegraded":"2024-03-29T04:09:22.656335+0000","last_fullsized":"2024-03-29T04:09:22.656335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329448+0000","last_change":"2024-03-29T04:09:16.307371+0000","last_active":"2024-03-29T04:09:22.329448+0000","last_peered":"2024-03-29T04:09:22.329448+0000","last_clean":"2024-03-29T04:09:22.329448+0000","last_became_active":"2024-03-29T04:09:16.307077+0000","last_became_peered":"2024-03-29T04:09:16.307077+0000","last_unstale":"2024-03-29T04:09:22.329448+0000","last_undegraded":"2024-03-29T04:09:22.329448+0000","last_fullsized":"2024-03-29T04:09:22.329448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328852+0000","last_change":"2024-03-29T04:09:20.883785+0000","last_active":"2024-03-29T04:09:22.328852+0000","last_peered":"2024-03-29T04:09:22.328852+0000","last_clean":"2024-03-29T04:09:22.328852+0000","last_became_active":"2024-03-29T04:09:20.883360+0000","last_became_peered":"2024-03-29T04:09:20.883360+0000","last_unstale":"2024-03-29T04:09:22.328852+0000","last_undegraded":"2024-03-29T04:09:22.328852+0000","last_fullsized":"2024-03-29T04:09:22.328852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.908030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329590+0000","last_change":"2024-03-29T04:09:15.301577+0000","last_active":"2024-03-29T04:09:22.329590+0000","last_peered":"2024-03-29T04:09:22.329590+0000","last_clean":"2024-03-29T04:09:22.329590+0000","last_became_active":"2024-03-29T04:09:15.301247+0000","last_became_peered":"2024-03-29T04:09:15.301247+0000","last_unstale":"2024-03-29T04:09:22.329590+0000","last_undegraded":"2024-03-29T04:09:22.329590+0000","last_fullsized":"2024-03-29T04:09:22.329590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656042+0000","last_change":"2024-03-29T04:09:16.319168+0000","last_active":"2024-03-29T04:09:22.656042+0000","last_peered":"2024-03-29T04:09:22.656042+0000","last_clean":"2024-03-29T04:09:22.656042+0000","last_became_active":"2024-03-29T04:09:16.318673+0000","last_became_peered":"2024-03-29T04:09:16.318673+0000","last_unstale":"2024-03-29T04:09:22.656042+0000","last_undegraded":"2024-03-29T04:09:22.656042+0000","last_fullsized":"2024-03-29T04:09:22.656042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:41.049934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329629+0000","last_change":"2024-03-29T04:09:20.348184+0000","last_active":"2024-03-29T04:09:22.329629+0000","last_peered":"2024-03-29T04:09:22.329629+0000","last_clean":"2024-03-29T04:09:22.329629+0000","last_became_active":"2024-03-29T04:09:20.347504+0000","last_became_peered":"2024-03-29T04:09:20.347504+0000","last_unstale":"2024-03-29T04:09:22.329629+0000","last_undegraded":"2024-03-29T04:09:22.329629+0000","last_fullsized":"2024-03-29T04:09:22.329629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656067+0000","last_change":"2024-03-29T04:09:15.308597+0000","last_active":"2024-03-29T04:09:22.656067+0000","last_peered":"2024-03-29T04:09:22.656067+0000","last_clean":"2024-03-29T04:09:22.656067+0000","last_became_active":"2024-03-29T04:09:15.308020+0000","last_became_peered":"2024-03-29T04:09:15.308020+0000","last_unstale":"2024-03-29T04:09:22.656067+0000","last_undegraded":"2024-03-29T04:09:22.656067+0000","last_fullsized":"2024-03-29T04:09:22.656067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:40.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655201+0000","last_change":"2024-03-29T04:09:16.321774+0000","last_active":"2024-03-29T04:09:22.655201+0000","last_peered":"2024-03-29T04:09:22.655201+0000","last_clean":"2024-03-29T04:09:22.655201+0000","last_became_active":"2024-03-29T04:09:16.321464+0000","last_became_peered":"2024-03-29T04:09:16.321464+0000","last_unstale":"2024-03-29T04:09:22.655201+0000","last_undegraded":"2024-03-29T04:09:22.655201+0000","last_fullsized":"2024-03-29T04:09:22.655201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330320+0000","last_change":"2024-03-29T04:09:20.346472+0000","last_active":"2024-03-29T04:09:22.330320+0000","last_peered":"2024-03-29T04:09:22.330320+0000","last_clean":"2024-03-29T04:09:22.330320+0000","last_became_active":"2024-03-29T04:09:20.346130+0000","last_became_peered":"2024-03-29T04:09:20.346130+0000","last_unstale":"2024-03-29T04:09:22.330320+0000","last_undegraded":"2024-03-29T04:09:22.330320+0000","last_fullsized":"2024-03-29T04:09:22.330320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655700+0000","last_change":"2024-03-29T04:09:15.308906+0000","last_active":"2024-03-29T04:09:22.655700+0000","last_peered":"2024-03-29T04:09:22.655700+0000","last_clean":"2024-03-29T04:09:22.655700+0000","last_became_active":"2024-03-29T04:09:15.308632+0000","last_became_peered":"2024-03-29T04:09:15.308632+0000","last_unstale":"2024-03-29T04:09:22.655700+0000","last_undegraded":"2024-03-29T04:09:22.655700+0000","last_fullsized":"2024-03-29T04:09:22.655700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.323173+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.322174+0000","last_became_peered":"2024-03-29T04:09:16.322174+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329721+0000","last_change":"2024-03-29T04:09:20.341688+0000","last_active":"2024-03-29T04:09:22.329721+0000","last_peered":"2024-03-29T04:09:22.329721+0000","last_clean":"2024-03-29T04:09:22.329721+0000","last_became_active":"2024-03-29T04:09:20.341328+0000","last_became_peered":"2024-03-29T04:09:20.341328+0000","last_unstale":"2024-03-29T04:09:22.329721+0000","last_undegraded":"2024-03-29T04:09:22.329721+0000","last_fullsized":"2024-03-29T04:09:22.329721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329443+0000","last_change":"2024-03-29T04:09:16.318229+0000","last_active":"2024-03-29T04:09:22.329443+0000","last_peered":"2024-03-29T04:09:22.329443+0000","last_clean":"2024-03-29T04:09:22.329443+0000","last_became_active":"2024-03-29T04:09:16.317967+0000","last_became_peered":"2024-03-29T04:09:16.317967+0000","last_unstale":"2024-03-29T04:09:22.329443+0000","last_undegraded":"2024-03-29T04:09:22.329443+0000","last_fullsized":"2024-03-29T04:09:22.329443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655679+0000","last_change":"2024-03-29T04:09:15.308430+0000","last_active":"2024-03-29T04:09:22.655679+0000","last_peered":"2024-03-29T04:09:22.655679+0000","last_clean":"2024-03-29T04:09:22.655679+0000","last_became_active":"2024-03-29T04:09:15.308131+0000","last_became_peered":"2024-03-29T04:09:15.308131+0000","last_unstale":"2024-03-29T04:09:22.655679+0000","last_undegraded":"2024-03-29T04:09:22.655679+0000","last_fullsized":"2024-03-29T04:09:22.655679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:36:32.401409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329454+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.329454+0000","last_peered":"2024-03-29T04:09:22.329454+0000","last_clean":"2024-03-29T04:09:22.329454+0000","last_became_active":"2024-03-29T04:09:20.347330+0000","last_became_peered":"2024-03-29T04:09:20.347330+0000","last_unstale":"2024-03-29T04:09:22.329454+0000","last_undegraded":"2024-03-29T04:09:22.329454+0000","last_fullsized":"2024-03-29T04:09:22.329454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329091+0000","last_change":"2024-03-29T04:09:16.311370+0000","last_active":"2024-03-29T04:09:22.329091+0000","last_peered":"2024-03-29T04:09:22.329091+0000","last_clean":"2024-03-29T04:09:22.329091+0000","last_became_active":"2024-03-29T04:09:16.311106+0000","last_became_peered":"2024-03-29T04:09:16.311106+0000","last_unstale":"2024-03-29T04:09:22.329091+0000","last_undegraded":"2024-03-29T04:09:22.329091+0000","last_fullsized":"2024-03-29T04:09:22.329091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656398+0000","last_change":"2024-03-29T04:09:15.306878+0000","last_active":"2024-03-29T04:09:22.656398+0000","last_peered":"2024-03-29T04:09:22.656398+0000","last_clean":"2024-03-29T04:09:22.656398+0000","last_became_active":"2024-03-29T04:09:15.306590+0000","last_became_peered":"2024-03-29T04:09:15.306590+0000","last_unstale":"2024-03-29T04:09:22.656398+0000","last_undegraded":"2024-03-29T04:09:22.656398+0000","last_fullsized":"2024-03-29T04:09:22.656398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330041+0000","last_change":"2024-03-29T04:09:20.347808+0000","last_active":"2024-03-29T04:09:22.330041+0000","last_peered":"2024-03-29T04:09:22.330041+0000","last_clean":"2024-03-29T04:09:22.330041+0000","last_became_active":"2024-03-29T04:09:20.347419+0000","last_became_peered":"2024-03-29T04:09:20.347419+0000","last_unstale":"2024-03-29T04:09:22.330041+0000","last_undegraded":"2024-03-29T04:09:22.330041+0000","last_fullsized":"2024-03-29T04:09:22.330041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:58.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328945+0000","last_change":"2024-03-29T04:09:16.309708+0000","last_active":"2024-03-29T04:09:22.328945+0000","last_peered":"2024-03-29T04:09:22.328945+0000","last_clean":"2024-03-29T04:09:22.328945+0000","last_became_active":"2024-03-29T04:09:16.309388+0000","last_became_peered":"2024-03-29T04:09:16.309388+0000","last_unstale":"2024-03-29T04:09:22.328945+0000","last_undegraded":"2024-03-29T04:09:22.328945+0000","last_fullsized":"2024-03-29T04:09:22.328945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655404+0000","last_change":"2024-03-29T04:09:15.308764+0000","last_active":"2024-03-29T04:09:22.655404+0000","last_peered":"2024-03-29T04:09:22.655404+0000","last_clean":"2024-03-29T04:09:22.655404+0000","last_became_active":"2024-03-29T04:09:15.308466+0000","last_became_peered":"2024-03-29T04:09:15.308466+0000","last_unstale":"2024-03-29T04:09:22.655404+0000","last_undegraded":"2024-03-29T04:09:22.655404+0000","last_fullsized":"2024-03-29T04:09:22.655404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657055+0000","last_change":"2024-03-29T04:09:20.883567+0000","last_active":"2024-03-29T04:09:22.657055+0000","last_peered":"2024-03-29T04:09:22.657055+0000","last_clean":"2024-03-29T04:09:22.657055+0000","last_became_active":"2024-03-29T04:09:20.883215+0000","last_became_peered":"2024-03-29T04:09:20.883215+0000","last_unstale":"2024-03-29T04:09:22.657055+0000","last_undegraded":"2024-03-29T04:09:22.657055+0000","last_fullsized":"2024-03-29T04:09:22.657055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330723+0000","last_change":"2024-03-29T04:09:16.307731+0000","last_active":"2024-03-29T04:09:22.330723+0000","last_peered":"2024-03-29T04:09:22.330723+0000","last_clean":"2024-03-29T04:09:22.330723+0000","last_became_active":"2024-03-29T04:09:16.307458+0000","last_became_peered":"2024-03-29T04:09:16.307458+0000","last_unstale":"2024-03-29T04:09:22.330723+0000","last_undegraded":"2024-03-29T04:09:22.330723+0000","last_fullsized":"2024-03-29T04:09:22.330723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:43.534030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654990+0000","last_change":"2024-03-29T04:09:15.306750+0000","last_active":"2024-03-29T04:09:22.654990+0000","last_peered":"2024-03-29T04:09:22.654990+0000","last_clean":"2024-03-29T04:09:22.654990+0000","last_became_active":"2024-03-29T04:09:15.306458+0000","last_became_peered":"2024-03-29T04:09:15.306458+0000","last_unstale":"2024-03-29T04:09:22.654990+0000","last_undegraded":"2024-03-29T04:09:22.654990+0000","last_fullsized":"2024-03-29T04:09:22.654990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330722+0000","last_change":"2024-03-29T04:09:20.346709+0000","last_active":"2024-03-29T04:09:22.330722+0000","last_peered":"2024-03-29T04:09:22.330722+0000","last_clean":"2024-03-29T04:09:22.330722+0000","last_became_active":"2024-03-29T04:09:20.346366+0000","last_became_peered":"2024-03-29T04:09:20.346366+0000","last_unstale":"2024-03-29T04:09:22.330722+0000","last_undegraded":"2024-03-29T04:09:22.330722+0000","last_fullsized":"2024-03-29T04:09:22.330722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656086+0000","last_change":"2024-03-29T04:09:16.311388+0000","last_active":"2024-03-29T04:09:22.656086+0000","last_peered":"2024-03-29T04:09:22.656086+0000","last_clean":"2024-03-29T04:09:22.656086+0000","last_became_active":"2024-03-29T04:09:16.311119+0000","last_became_peered":"2024-03-29T04:09:16.311119+0000","last_unstale":"2024-03-29T04:09:22.656086+0000","last_undegraded":"2024-03-29T04:09:22.656086+0000","last_fullsized":"2024-03-29T04:09:22.656086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330110+0000","last_change":"2024-03-29T04:09:15.301237+0000","last_active":"2024-03-29T04:09:22.330110+0000","last_peered":"2024-03-29T04:09:22.330110+0000","last_clean":"2024-03-29T04:09:22.330110+0000","last_became_active":"2024-03-29T04:09:15.300671+0000","last_became_peered":"2024-03-29T04:09:15.300671+0000","last_unstale":"2024-03-29T04:09:22.330110+0000","last_undegraded":"2024-03-29T04:09:22.330110+0000","last_fullsized":"2024-03-29T04:09:22.330110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330650+0000","last_change":"2024-03-29T04:09:20.882096+0000","last_active":"2024-03-29T04:09:22.330650+0000","last_peered":"2024-03-29T04:09:22.330650+0000","last_clean":"2024-03-29T04:09:22.330650+0000","last_became_active":"2024-03-29T04:09:20.881672+0000","last_became_peered":"2024-03-29T04:09:20.881672+0000","last_unstale":"2024-03-29T04:09:22.330650+0000","last_undegraded":"2024-03-29T04:09:22.330650+0000","last_fullsized":"2024-03-29T04:09:22.330650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329679+0000","last_change":"2024-03-29T04:09:16.308204+0000","last_active":"2024-03-29T04:09:22.329679+0000","last_peered":"2024-03-29T04:09:22.329679+0000","last_clean":"2024-03-29T04:09:22.329679+0000","last_became_active":"2024-03-29T04:09:16.307931+0000","last_became_peered":"2024-03-29T04:09:16.307931+0000","last_unstale":"2024-03-29T04:09:22.329679+0000","last_undegraded":"2024-03-29T04:09:22.329679+0000","last_fullsized":"2024-03-29T04:09:22.329679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656940+0000","last_change":"2024-03-29T04:09:15.303590+0000","last_active":"2024-03-29T04:09:22.656940+0000","last_peered":"2024-03-29T04:09:22.656940+0000","last_clean":"2024-03-29T04:09:22.656940+0000","last_became_active":"2024-03-29T04:09:15.303295+0000","last_became_peered":"2024-03-29T04:09:15.303295+0000","last_unstale":"2024-03-29T04:09:22.656940+0000","last_undegraded":"2024-03-29T04:09:22.656940+0000","last_fullsized":"2024-03-29T04:09:22.656940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328789+0000","last_change":"2024-03-29T04:09:20.884166+0000","last_active":"2024-03-29T04:09:22.328789+0000","last_peered":"2024-03-29T04:09:22.328789+0000","last_clean":"2024-03-29T04:09:22.328789+0000","last_became_active":"2024-03-29T04:09:20.883801+0000","last_became_peered":"2024-03-29T04:09:20.883801+0000","last_unstale":"2024-03-29T04:09:22.328789+0000","last_undegraded":"2024-03-29T04:09:22.328789+0000","last_fullsized":"2024-03-29T04:09:22.328789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:45.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656397+0000","last_change":"2024-03-29T04:09:16.307299+0000","last_active":"2024-03-29T04:09:22.656397+0000","last_peered":"2024-03-29T04:09:22.656397+0000","last_clean":"2024-03-29T04:09:22.656397+0000","last_became_active":"2024-03-29T04:09:16.306982+0000","last_became_peered":"2024-03-29T04:09:16.306982+0000","last_unstale":"2024-03-29T04:09:22.656397+0000","last_undegraded":"2024-03-29T04:09:22.656397+0000","last_fullsized":"2024-03-29T04:09:22.656397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656158+0000","last_change":"2024-03-29T04:09:15.308823+0000","last_active":"2024-03-29T04:09:22.656158+0000","last_peered":"2024-03-29T04:09:22.656158+0000","last_clean":"2024-03-29T04:09:22.656158+0000","last_became_active":"2024-03-29T04:09:15.308335+0000","last_became_peered":"2024-03-29T04:09:15.308335+0000","last_unstale":"2024-03-29T04:09:22.656158+0000","last_undegraded":"2024-03-29T04:09:22.656158+0000","last_fullsized":"2024-03-29T04:09:22.656158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:09.173137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655324+0000","last_change":"2024-03-29T04:09:20.882760+0000","last_active":"2024-03-29T04:09:22.655324+0000","last_peered":"2024-03-29T04:09:22.655324+0000","last_clean":"2024-03-29T04:09:22.655324+0000","last_became_active":"2024-03-29T04:09:20.881819+0000","last_became_peered":"2024-03-29T04:09:20.881819+0000","last_unstale":"2024-03-29T04:09:22.655324+0000","last_undegraded":"2024-03-29T04:09:22.655324+0000","last_fullsized":"2024-03-29T04:09:22.655324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655417+0000","last_change":"2024-03-29T04:09:16.318039+0000","last_active":"2024-03-29T04:09:22.655417+0000","last_peered":"2024-03-29T04:09:22.655417+0000","last_clean":"2024-03-29T04:09:22.655417+0000","last_became_active":"2024-03-29T04:09:16.317784+0000","last_became_peered":"2024-03-29T04:09:16.317784+0000","last_unstale":"2024-03-29T04:09:22.655417+0000","last_undegraded":"2024-03-29T04:09:22.655417+0000","last_fullsized":"2024-03-29T04:09:22.655417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656459+0000","last_change":"2024-03-29T04:09:15.302818+0000","last_active":"2024-03-29T04:09:22.656459+0000","last_peered":"2024-03-29T04:09:22.656459+0000","last_clean":"2024-03-29T04:09:22.656459+0000","last_became_active":"2024-03-29T04:09:15.302521+0000","last_became_peered":"2024-03-29T04:09:15.302521+0000","last_unstale":"2024-03-29T04:09:22.656459+0000","last_undegraded":"2024-03-29T04:09:22.656459+0000","last_fullsized":"2024-03-29T04:09:22.656459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656072+0000","last_change":"2024-03-29T04:09:20.351951+0000","last_active":"2024-03-29T04:09:22.656072+0000","last_peered":"2024-03-29T04:09:22.656072+0000","last_clean":"2024-03-29T04:09:22.656072+0000","last_became_active":"2024-03-29T04:09:20.351337+0000","last_became_peered":"2024-03-29T04:09:20.351337+0000","last_unstale":"2024-03-29T04:09:22.656072+0000","last_undegraded":"2024-03-29T04:09:22.656072+0000","last_fullsized":"2024-03-29T04:09:22.656072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330495+0000","last_change":"2024-03-29T04:09:16.316074+0000","last_active":"2024-03-29T04:09:22.330495+0000","last_peered":"2024-03-29T04:09:22.330495+0000","last_clean":"2024-03-29T04:09:22.330495+0000","last_became_active":"2024-03-29T04:09:16.315644+0000","last_became_peered":"2024-03-29T04:09:16.315644+0000","last_unstale":"2024-03-29T04:09:22.330495+0000","last_undegraded":"2024-03-29T04:09:22.330495+0000","last_fullsized":"2024-03-29T04:09:22.330495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:13.041449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655320+0000","last_change":"2024-03-29T04:09:15.307707+0000","last_active":"2024-03-29T04:09:22.655320+0000","last_peered":"2024-03-29T04:09:22.655320+0000","last_clean":"2024-03-29T04:09:22.655320+0000","last_became_active":"2024-03-29T04:09:15.307411+0000","last_became_peered":"2024-03-29T04:09:15.307411+0000","last_unstale":"2024-03-29T04:09:22.655320+0000","last_undegraded":"2024-03-29T04:09:22.655320+0000","last_fullsized":"2024-03-29T04:09:22.655320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656552+0000","last_change":"2024-03-29T04:09:20.346773+0000","last_active":"2024-03-29T04:09:22.656552+0000","last_peered":"2024-03-29T04:09:22.656552+0000","last_clean":"2024-03-29T04:09:22.656552+0000","last_became_active":"2024-03-29T04:09:20.346383+0000","last_became_peered":"2024-03-29T04:09:20.346383+0000","last_unstale":"2024-03-29T04:09:22.656552+0000","last_undegraded":"2024-03-29T04:09:22.656552+0000","last_fullsized":"2024-03-29T04:09:22.656552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.311447+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.311171+0000","last_became_peered":"2024-03-29T04:09:16.311171+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656334+0000","last_change":"2024-03-29T04:09:15.307046+0000","last_active":"2024-03-29T04:09:22.656334+0000","last_peered":"2024-03-29T04:09:22.656334+0000","last_clean":"2024-03-29T04:09:22.656334+0000","last_became_active":"2024-03-29T04:09:15.305761+0000","last_became_peered":"2024-03-29T04:09:15.305761+0000","last_unstale":"2024-03-29T04:09:22.656334+0000","last_undegraded":"2024-03-29T04:09:22.656334+0000","last_fullsized":"2024-03-29T04:09:22.656334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657098+0000","last_change":"2024-03-29T04:09:20.346732+0000","last_active":"2024-03-29T04:09:22.657098+0000","last_peered":"2024-03-29T04:09:22.657098+0000","last_clean":"2024-03-29T04:09:22.657098+0000","last_became_active":"2024-03-29T04:09:20.346121+0000","last_became_peered":"2024-03-29T04:09:20.346121+0000","last_unstale":"2024-03-29T04:09:22.657098+0000","last_undegraded":"2024-03-29T04:09:22.657098+0000","last_fullsized":"2024-03-29T04:09:22.657098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655371+0000","last_change":"2024-03-29T04:09:16.318196+0000","last_active":"2024-03-29T04:09:22.655371+0000","last_peered":"2024-03-29T04:09:22.655371+0000","last_clean":"2024-03-29T04:09:22.655371+0000","last_became_active":"2024-03-29T04:09:16.317906+0000","last_became_peered":"2024-03-29T04:09:16.317906+0000","last_unstale":"2024-03-29T04:09:22.655371+0000","last_undegraded":"2024-03-29T04:09:22.655371+0000","last_fullsized":"2024-03-29T04:09:22.655371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:36.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655070+0000","last_change":"2024-03-29T04:09:15.308083+0000","last_active":"2024-03-29T04:09:22.655070+0000","last_peered":"2024-03-29T04:09:22.655070+0000","last_clean":"2024-03-29T04:09:22.655070+0000","last_became_active":"2024-03-29T04:09:15.307813+0000","last_became_peered":"2024-03-29T04:09:15.307813+0000","last_unstale":"2024-03-29T04:09:22.655070+0000","last_undegraded":"2024-03-29T04:09:22.655070+0000","last_fullsized":"2024-03-29T04:09:22.655070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655909+0000","last_change":"2024-03-29T04:09:20.882216+0000","last_active":"2024-03-29T04:09:22.655909+0000","last_peered":"2024-03-29T04:09:22.655909+0000","last_clean":"2024-03-29T04:09:22.655909+0000","last_became_active":"2024-03-29T04:09:20.881801+0000","last_became_peered":"2024-03-29T04:09:20.881801+0000","last_unstale":"2024-03-29T04:09:22.655909+0000","last_undegraded":"2024-03-29T04:09:22.655909+0000","last_fullsized":"2024-03-29T04:09:22.655909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328920+0000","last_change":"2024-03-29T04:09:16.319621+0000","last_active":"2024-03-29T04:09:22.328920+0000","last_peered":"2024-03-29T04:09:22.328920+0000","last_clean":"2024-03-29T04:09:22.328920+0000","last_became_active":"2024-03-29T04:09:16.318978+0000","last_became_peered":"2024-03-29T04:09:16.318978+0000","last_unstale":"2024-03-29T04:09:22.328920+0000","last_undegraded":"2024-03-29T04:09:22.328920+0000","last_fullsized":"2024-03-29T04:09:22.328920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655084+0000","last_change":"2024-03-29T04:09:15.302638+0000","last_active":"2024-03-29T04:09:22.655084+0000","last_peered":"2024-03-29T04:09:22.655084+0000","last_clean":"2024-03-29T04:09:22.655084+0000","last_became_active":"2024-03-29T04:09:15.302211+0000","last_became_peered":"2024-03-29T04:09:15.302211+0000","last_unstale":"2024-03-29T04:09:22.655084+0000","last_undegraded":"2024-03-29T04:09:22.655084+0000","last_fullsized":"2024-03-29T04:09:22.655084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655145+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:09:22.655145+0000","last_peered":"2024-03-29T04:09:22.655145+0000","last_clean":"2024-03-29T04:09:22.655145+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:09:22.655145+0000","last_undegraded":"2024-03-29T04:09:22.655145+0000","last_fullsized":"2024-03-29T04:09:22.655145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329351+0000","last_change":"2024-03-29T04:09:20.883146+0000","last_active":"2024-03-29T04:09:22.329351+0000","last_peered":"2024-03-29T04:09:22.329351+0000","last_clean":"2024-03-29T04:09:22.329351+0000","last_became_active":"2024-03-29T04:09:20.882795+0000","last_became_peered":"2024-03-29T04:09:20.882795+0000","last_unstale":"2024-03-29T04:09:22.329351+0000","last_undegraded":"2024-03-29T04:09:22.329351+0000","last_fullsized":"2024-03-29T04:09:22.329351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655937+0000","last_change":"2024-03-29T04:09:16.316245+0000","last_active":"2024-03-29T04:09:22.655937+0000","last_peered":"2024-03-29T04:09:22.655937+0000","last_clean":"2024-03-29T04:09:22.655937+0000","last_became_active":"2024-03-29T04:09:16.315972+0000","last_became_peered":"2024-03-29T04:09:16.315972+0000","last_unstale":"2024-03-29T04:09:22.655937+0000","last_undegraded":"2024-03-29T04:09:22.655937+0000","last_fullsized":"2024-03-29T04:09:22.655937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656723+0000","last_change":"2024-03-29T04:09:15.304758+0000","last_active":"2024-03-29T04:09:22.656723+0000","last_peered":"2024-03-29T04:09:22.656723+0000","last_clean":"2024-03-29T04:09:22.656723+0000","last_became_active":"2024-03-29T04:09:15.304450+0000","last_became_peered":"2024-03-29T04:09:15.304450+0000","last_unstale":"2024-03-29T04:09:22.656723+0000","last_undegraded":"2024-03-29T04:09:22.656723+0000","last_fullsized":"2024-03-29T04:09:22.656723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329391+0000","last_change":"2024-03-29T04:09:20.882224+0000","last_active":"2024-03-29T04:09:22.329391+0000","last_peered":"2024-03-29T04:09:22.329391+0000","last_clean":"2024-03-29T04:09:22.329391+0000","last_became_active":"2024-03-29T04:09:20.881813+0000","last_became_peered":"2024-03-29T04:09:20.881813+0000","last_unstale":"2024-03-29T04:09:22.329391+0000","last_undegraded":"2024-03-29T04:09:22.329391+0000","last_fullsized":"2024-03-29T04:09:22.329391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655284+0000","last_change":"2024-03-29T04:09:16.322983+0000","last_active":"2024-03-29T04:09:22.655284+0000","last_peered":"2024-03-29T04:09:22.655284+0000","last_clean":"2024-03-29T04:09:22.655284+0000","last_became_active":"2024-03-29T04:09:16.322712+0000","last_became_peered":"2024-03-29T04:09:16.322712+0000","last_unstale":"2024-03-29T04:09:22.655284+0000","last_undegraded":"2024-03-29T04:09:22.655284+0000","last_fullsized":"2024-03-29T04:09:22.655284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:15.302835+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:15.302365+0000","last_became_peered":"2024-03-29T04:09:15.302365+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655981+0000","last_change":"2024-03-29T04:09:20.347857+0000","last_active":"2024-03-29T04:09:22.655981+0000","last_peered":"2024-03-29T04:09:22.655981+0000","last_clean":"2024-03-29T04:09:22.655981+0000","last_became_active":"2024-03-29T04:09:20.347499+0000","last_became_peered":"2024-03-29T04:09:20.347499+0000","last_unstale":"2024-03-29T04:09:22.655981+0000","last_undegraded":"2024-03-29T04:09:22.655981+0000","last_fullsized":"2024-03-29T04:09:22.655981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656262+0000","last_change":"2024-03-29T04:09:16.321770+0000","last_active":"2024-03-29T04:09:22.656262+0000","last_peered":"2024-03-29T04:09:22.656262+0000","last_clean":"2024-03-29T04:09:22.656262+0000","last_became_active":"2024-03-29T04:09:16.321511+0000","last_became_peered":"2024-03-29T04:09:16.321511+0000","last_unstale":"2024-03-29T04:09:22.656262+0000","last_undegraded":"2024-03-29T04:09:22.656262+0000","last_fullsized":"2024-03-29T04:09:22.656262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656031+0000","last_change":"2024-03-29T04:09:15.306618+0000","last_active":"2024-03-29T04:09:22.656031+0000","last_peered":"2024-03-29T04:09:22.656031+0000","last_clean":"2024-03-29T04:09:22.656031+0000","last_became_active":"2024-03-29T04:09:15.306337+0000","last_became_peered":"2024-03-29T04:09:15.306337+0000","last_unstale":"2024-03-29T04:09:22.656031+0000","last_undegraded":"2024-03-29T04:09:22.656031+0000","last_fullsized":"2024-03-29T04:09:22.656031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654617+0000","last_change":"2024-03-29T04:09:20.884325+0000","last_active":"2024-03-29T04:09:22.654617+0000","last_peered":"2024-03-29T04:09:22.654617+0000","last_clean":"2024-03-29T04:09:22.654617+0000","last_became_active":"2024-03-29T04:09:20.883509+0000","last_became_peered":"2024-03-29T04:09:20.883509+0000","last_unstale":"2024-03-29T04:09:22.654617+0000","last_undegraded":"2024-03-29T04:09:22.654617+0000","last_fullsized":"2024-03-29T04:09:22.654617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:45:02.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655965+0000","last_change":"2024-03-29T04:09:16.309527+0000","last_active":"2024-03-29T04:09:22.655965+0000","last_peered":"2024-03-29T04:09:22.655965+0000","last_clean":"2024-03-29T04:09:22.655965+0000","last_became_active":"2024-03-29T04:09:16.309219+0000","last_became_peered":"2024-03-29T04:09:16.309219+0000","last_unstale":"2024-03-29T04:09:22.655965+0000","last_undegraded":"2024-03-29T04:09:22.655965+0000","last_fullsized":"2024-03-29T04:09:22.655965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329782+0000","last_change":"2024-03-29T04:09:15.306652+0000","last_active":"2024-03-29T04:09:22.329782+0000","last_peered":"2024-03-29T04:09:22.329782+0000","last_clean":"2024-03-29T04:09:22.329782+0000","last_became_active":"2024-03-29T04:09:15.306354+0000","last_became_peered":"2024-03-29T04:09:15.306354+0000","last_unstale":"2024-03-29T04:09:22.329782+0000","last_undegraded":"2024-03-29T04:09:22.329782+0000","last_fullsized":"2024-03-29T04:09:22.329782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330540+0000","last_change":"2024-03-29T04:09:20.346199+0000","last_active":"2024-03-29T04:09:22.330540+0000","last_peered":"2024-03-29T04:09:22.330540+0000","last_clean":"2024-03-29T04:09:22.330540+0000","last_became_active":"2024-03-29T04:09:20.345826+0000","last_became_peered":"2024-03-29T04:09:20.345826+0000","last_unstale":"2024-03-29T04:09:22.330540+0000","last_undegraded":"2024-03-29T04:09:22.330540+0000","last_fullsized":"2024-03-29T04:09:22.330540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.321785+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.321533+0000","last_became_peered":"2024-03-29T04:09:16.321533+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655385+0000","last_change":"2024-03-29T04:09:15.307664+0000","last_active":"2024-03-29T04:09:22.655385+0000","last_peered":"2024-03-29T04:09:22.655385+0000","last_clean":"2024-03-29T04:09:22.655385+0000","last_became_active":"2024-03-29T04:09:15.307412+0000","last_became_peered":"2024-03-29T04:09:15.307412+0000","last_unstale":"2024-03-29T04:09:22.655385+0000","last_undegraded":"2024-03-29T04:09:22.655385+0000","last_fullsized":"2024-03-29T04:09:22.655385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655742+0000","last_change":"2024-03-29T04:09:20.346476+0000","last_active":"2024-03-29T04:09:22.655742+0000","last_peered":"2024-03-29T04:09:22.655742+0000","last_clean":"2024-03-29T04:09:22.655742+0000","last_became_active":"2024-03-29T04:09:20.346140+0000","last_became_peered":"2024-03-29T04:09:20.346140+0000","last_unstale":"2024-03-29T04:09:22.655742+0000","last_undegraded":"2024-03-29T04:09:22.655742+0000","last_fullsized":"2024-03-29T04:09:22.655742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329068+0000","last_change":"2024-03-29T04:09:16.313068+0000","last_active":"2024-03-29T04:09:22.329068+0000","last_peered":"2024-03-29T04:09:22.329068+0000","last_clean":"2024-03-29T04:09:22.329068+0000","last_became_active":"2024-03-29T04:09:16.312754+0000","last_became_peered":"2024-03-29T04:09:16.312754+0000","last_unstale":"2024-03-29T04:09:22.329068+0000","last_undegraded":"2024-03-29T04:09:22.329068+0000","last_fullsized":"2024-03-29T04:09:22.329068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329238+0000","last_change":"2024-03-29T04:09:15.300287+0000","last_active":"2024-03-29T04:09:22.329238+0000","last_peered":"2024-03-29T04:09:22.329238+0000","last_clean":"2024-03-29T04:09:22.329238+0000","last_became_active":"2024-03-29T04:09:15.299961+0000","last_became_peered":"2024-03-29T04:09:15.299961+0000","last_unstale":"2024-03-29T04:09:22.329238+0000","last_undegraded":"2024-03-29T04:09:22.329238+0000","last_fullsized":"2024-03-29T04:09:22.329238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329627+0000","last_change":"2024-03-29T04:09:20.883520+0000","last_active":"2024-03-29T04:09:22.329627+0000","last_peered":"2024-03-29T04:09:22.329627+0000","last_clean":"2024-03-29T04:09:22.329627+0000","last_became_active":"2024-03-29T04:09:20.883152+0000","last_became_peered":"2024-03-29T04:09:20.883152+0000","last_unstale":"2024-03-29T04:09:22.329627+0000","last_undegraded":"2024-03-29T04:09:22.329627+0000","last_fullsized":"2024-03-29T04:09:22.329627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:25.394492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655644+0000","last_change":"2024-03-29T04:09:16.322576+0000","last_active":"2024-03-29T04:09:22.655644+0000","last_peered":"2024-03-29T04:09:22.655644+0000","last_clean":"2024-03-29T04:09:22.655644+0000","last_became_active":"2024-03-29T04:09:16.322126+0000","last_became_peered":"2024-03-29T04:09:16.322126+0000","last_unstale":"2024-03-29T04:09:22.655644+0000","last_undegraded":"2024-03-29T04:09:22.655644+0000","last_fullsized":"2024-03-29T04:09:22.655644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655221+0000","last_change":"2024-03-29T04:09:15.306114+0000","last_active":"2024-03-29T04:09:22.655221+0000","last_peered":"2024-03-29T04:09:22.655221+0000","last_clean":"2024-03-29T04:09:22.655221+0000","last_became_active":"2024-03-29T04:09:15.305893+0000","last_became_peered":"2024-03-29T04:09:15.305893+0000","last_unstale":"2024-03-29T04:09:22.655221+0000","last_undegraded":"2024-03-29T04:09:22.655221+0000","last_fullsized":"2024-03-29T04:09:22.655221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329246+0000","last_change":"2024-03-29T04:09:20.882277+0000","last_active":"2024-03-29T04:09:22.329246+0000","last_peered":"2024-03-29T04:09:22.329246+0000","last_clean":"2024-03-29T04:09:22.329246+0000","last_became_active":"2024-03-29T04:09:20.881878+0000","last_became_peered":"2024-03-29T04:09:20.881878+0000","last_unstale":"2024-03-29T04:09:22.329246+0000","last_undegraded":"2024-03-29T04:09:22.329246+0000","last_fullsized":"2024-03-29T04:09:22.329246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:06.012938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655785+0000","last_change":"2024-03-29T04:09:16.307882+0000","last_active":"2024-03-29T04:09:22.655785+0000","last_peered":"2024-03-29T04:09:22.655785+0000","last_clean":"2024-03-29T04:09:22.655785+0000","last_became_active":"2024-03-29T04:09:16.307620+0000","last_became_peered":"2024-03-29T04:09:16.307620+0000","last_unstale":"2024-03-29T04:09:22.655785+0000","last_undegraded":"2024-03-29T04:09:22.655785+0000","last_fullsized":"2024-03-29T04:09:22.655785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656164+0000","last_change":"2024-03-29T04:09:15.300985+0000","last_active":"2024-03-29T04:09:22.656164+0000","last_peered":"2024-03-29T04:09:22.656164+0000","last_clean":"2024-03-29T04:09:22.656164+0000","last_became_active":"2024-03-29T04:09:15.300603+0000","last_became_peered":"2024-03-29T04:09:15.300603+0000","last_unstale":"2024-03-29T04:09:22.656164+0000","last_undegraded":"2024-03-29T04:09:22.656164+0000","last_fullsized":"2024-03-29T04:09:22.656164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:12.516429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655363+0000","last_change":"2024-03-29T04:09:20.883953+0000","last_active":"2024-03-29T04:09:22.655363+0000","last_peered":"2024-03-29T04:09:22.655363+0000","last_clean":"2024-03-29T04:09:22.655363+0000","last_became_active":"2024-03-29T04:09:20.883597+0000","last_became_peered":"2024-03-29T04:09:20.883597+0000","last_unstale":"2024-03-29T04:09:22.655363+0000","last_undegraded":"2024-03-29T04:09:22.655363+0000","last_fullsized":"2024-03-29T04:09:22.655363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:27.295430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655864+0000","last_change":"2024-03-29T04:09:16.306905+0000","last_active":"2024-03-29T04:09:22.655864+0000","last_peered":"2024-03-29T04:09:22.655864+0000","last_clean":"2024-03-29T04:09:22.655864+0000","last_became_active":"2024-03-29T04:09:16.306156+0000","last_became_peered":"2024-03-29T04:09:16.306156+0000","last_unstale":"2024-03-29T04:09:22.655864+0000","last_undegraded":"2024-03-29T04:09:22.655864+0000","last_fullsized":"2024-03-29T04:09:22.655864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:25.361360+0000","last_change":"2024-03-29T04:09:15.302359+0000","last_active":"2024-03-29T04:09:25.361360+0000","last_peered":"2024-03-29T04:09:25.361360+0000","last_clean":"2024-03-29T04:09:25.361360+0000","last_became_active":"2024-03-29T04:09:15.302044+0000","last_became_peered":"2024-03-29T04:09:15.302044+0000","last_unstale":"2024-03-29T04:09:25.361360+0000","last_undegraded":"2024-03-29T04:09:25.361360+0000","last_fullsized":"2024-03-29T04:09:25.361360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656202+0000","last_change":"2024-03-29T04:09:20.352101+0000","last_active":"2024-03-29T04:09:22.656202+0000","last_peered":"2024-03-29T04:09:22.656202+0000","last_clean":"2024-03-29T04:09:22.656202+0000","last_became_active":"2024-03-29T04:09:20.351597+0000","last_became_peered":"2024-03-29T04:09:20.351597+0000","last_unstale":"2024-03-29T04:09:22.656202+0000","last_undegraded":"2024-03-29T04:09:22.656202+0000","last_fullsized":"2024-03-29T04:09:22.656202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330245+0000","last_change":"2024-03-29T04:09:16.305800+0000","last_active":"2024-03-29T04:09:22.330245+0000","last_peered":"2024-03-29T04:09:22.330245+0000","last_clean":"2024-03-29T04:09:22.330245+0000","last_became_active":"2024-03-29T04:09:16.305534+0000","last_became_peered":"2024-03-29T04:09:16.305534+0000","last_unstale":"2024-03-29T04:09:22.330245+0000","last_undegraded":"2024-03-29T04:09:22.330245+0000","last_fullsized":"2024-03-29T04:09:22.330245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655674+0000","last_change":"2024-03-29T04:09:15.309193+0000","last_active":"2024-03-29T04:09:22.655674+0000","last_peered":"2024-03-29T04:09:22.655674+0000","last_clean":"2024-03-29T04:09:22.655674+0000","last_became_active":"2024-03-29T04:09:15.308884+0000","last_became_peered":"2024-03-29T04:09:15.308884+0000","last_unstale":"2024-03-29T04:09:22.655674+0000","last_undegraded":"2024-03-29T04:09:22.655674+0000","last_fullsized":"2024-03-29T04:09:22.655674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330135+0000","last_change":"2024-03-29T04:09:20.348029+0000","last_active":"2024-03-29T04:09:22.330135+0000","last_peered":"2024-03-29T04:09:22.330135+0000","last_clean":"2024-03-29T04:09:22.330135+0000","last_became_active":"2024-03-29T04:09:20.347196+0000","last_became_peered":"2024-03-29T04:09:20.347196+0000","last_unstale":"2024-03-29T04:09:22.330135+0000","last_undegraded":"2024-03-29T04:09:22.330135+0000","last_fullsized":"2024-03-29T04:09:22.330135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656194+0000","last_change":"2024-03-29T04:09:16.318939+0000","last_active":"2024-03-29T04:09:22.656194+0000","last_peered":"2024-03-29T04:09:22.656194+0000","last_clean":"2024-03-29T04:09:22.656194+0000","last_became_active":"2024-03-29T04:09:16.318327+0000","last_became_peered":"2024-03-29T04:09:16.318327+0000","last_unstale":"2024-03-29T04:09:22.656194+0000","last_undegraded":"2024-03-29T04:09:22.656194+0000","last_fullsized":"2024-03-29T04:09:22.656194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654888+0000","last_change":"2024-03-29T04:09:15.300878+0000","last_active":"2024-03-29T04:09:22.654888+0000","last_peered":"2024-03-29T04:09:22.654888+0000","last_clean":"2024-03-29T04:09:22.654888+0000","last_became_active":"2024-03-29T04:09:15.300567+0000","last_became_peered":"2024-03-29T04:09:15.300567+0000","last_unstale":"2024-03-29T04:09:22.654888+0000","last_undegraded":"2024-03-29T04:09:22.654888+0000","last_fullsized":"2024-03-29T04:09:22.654888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328959+0000","last_change":"2024-03-29T04:09:20.343510+0000","last_active":"2024-03-29T04:09:22.328959+0000","last_peered":"2024-03-29T04:09:22.328959+0000","last_clean":"2024-03-29T04:09:22.328959+0000","last_became_active":"2024-03-29T04:09:20.343070+0000","last_became_peered":"2024-03-29T04:09:20.343070+0000","last_unstale":"2024-03-29T04:09:22.328959+0000","last_undegraded":"2024-03-29T04:09:22.328959+0000","last_fullsized":"2024-03-29T04:09:22.328959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:10.010570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654868+0000","last_change":"2024-03-29T04:09:16.314479+0000","last_active":"2024-03-29T04:09:22.654868+0000","last_peered":"2024-03-29T04:09:22.654868+0000","last_clean":"2024-03-29T04:09:22.654868+0000","last_became_active":"2024-03-29T04:09:16.313990+0000","last_became_peered":"2024-03-29T04:09:16.313990+0000","last_unstale":"2024-03-29T04:09:22.654868+0000","last_undegraded":"2024-03-29T04:09:22.654868+0000","last_fullsized":"2024-03-29T04:09:22.654868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329948+0000","last_change":"2024-03-29T04:09:15.304665+0000","last_active":"2024-03-29T04:09:22.329948+0000","last_peered":"2024-03-29T04:09:22.329948+0000","last_clean":"2024-03-29T04:09:22.329948+0000","last_became_active":"2024-03-29T04:09:15.304381+0000","last_became_peered":"2024-03-29T04:09:15.304381+0000","last_unstale":"2024-03-29T04:09:22.329948+0000","last_undegraded":"2024-03-29T04:09:22.329948+0000","last_fullsized":"2024-03-29T04:09:22.329948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329035+0000","last_change":"2024-03-29T04:09:20.884477+0000","last_active":"2024-03-29T04:09:22.329035+0000","last_peered":"2024-03-29T04:09:22.329035+0000","last_clean":"2024-03-29T04:09:22.329035+0000","last_became_active":"2024-03-29T04:09:20.884123+0000","last_became_peered":"2024-03-29T04:09:20.884123+0000","last_unstale":"2024-03-29T04:09:22.329035+0000","last_undegraded":"2024-03-29T04:09:22.329035+0000","last_fullsized":"2024-03-29T04:09:22.329035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:11.194987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655599+0000","last_change":"2024-03-29T04:09:20.883017+0000","last_active":"2024-03-29T04:09:22.655599+0000","last_peered":"2024-03-29T04:09:22.655599+0000","last_clean":"2024-03-29T04:09:22.655599+0000","last_became_active":"2024-03-29T04:09:20.882655+0000","last_became_peered":"2024-03-29T04:09:20.882655+0000","last_unstale":"2024-03-29T04:09:22.655599+0000","last_undegraded":"2024-03-29T04:09:22.655599+0000","last_fullsized":"2024-03-29T04:09:22.655599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655207+0000","last_change":"2024-03-29T04:09:16.313418+0000","last_active":"2024-03-29T04:09:22.655207+0000","last_peered":"2024-03-29T04:09:22.655207+0000","last_clean":"2024-03-29T04:09:22.655207+0000","last_became_active":"2024-03-29T04:09:16.312923+0000","last_became_peered":"2024-03-29T04:09:16.312923+0000","last_unstale":"2024-03-29T04:09:22.655207+0000","last_undegraded":"2024-03-29T04:09:22.655207+0000","last_fullsized":"2024-03-29T04:09:22.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654732+0000","last_change":"2024-03-29T04:09:15.304449+0000","last_active":"2024-03-29T04:09:22.654732+0000","last_peered":"2024-03-29T04:09:22.654732+0000","last_clean":"2024-03-29T04:09:22.654732+0000","last_became_active":"2024-03-29T04:09:15.304164+0000","last_became_peered":"2024-03-29T04:09:15.304164+0000","last_unstale":"2024-03-29T04:09:22.654732+0000","last_undegraded":"2024-03-29T04:09:22.654732+0000","last_fullsized":"2024-03-29T04:09:22.654732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329921+0000","last_change":"2024-03-29T04:09:20.344669+0000","last_active":"2024-03-29T04:09:22.329921+0000","last_peered":"2024-03-29T04:09:22.329921+0000","last_clean":"2024-03-29T04:09:22.329921+0000","last_became_active":"2024-03-29T04:09:20.344359+0000","last_became_peered":"2024-03-29T04:09:20.344359+0000","last_unstale":"2024-03-29T04:09:22.329921+0000","last_undegraded":"2024-03-29T04:09:22.329921+0000","last_fullsized":"2024-03-29T04:09:22.329921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328865+0000","last_change":"2024-03-29T04:09:16.308421+0000","last_active":"2024-03-29T04:09:22.328865+0000","last_peered":"2024-03-29T04:09:22.328865+0000","last_clean":"2024-03-29T04:09:22.328865+0000","last_became_active":"2024-03-29T04:09:16.308142+0000","last_became_peered":"2024-03-29T04:09:16.308142+0000","last_unstale":"2024-03-29T04:09:22.328865+0000","last_undegraded":"2024-03-29T04:09:22.328865+0000","last_fullsized":"2024-03-29T04:09:22.328865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329237+0000","last_change":"2024-03-29T04:09:15.302691+0000","last_active":"2024-03-29T04:09:22.329237+0000","last_peered":"2024-03-29T04:09:22.329237+0000","last_clean":"2024-03-29T04:09:22.329237+0000","last_became_active":"2024-03-29T04:09:15.302305+0000","last_became_peered":"2024-03-29T04:09:15.302305+0000","last_unstale":"2024-03-29T04:09:22.329237+0000","last_undegraded":"2024-03-29T04:09:22.329237+0000","last_fullsized":"2024-03-29T04:09:22.329237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329606+0000","last_change":"2024-03-29T04:09:15.299655+0000","last_active":"2024-03-29T04:09:22.329606+0000","last_peered":"2024-03-29T04:09:22.329606+0000","last_clean":"2024-03-29T04:09:22.329606+0000","last_became_active":"2024-03-29T04:09:15.299361+0000","last_became_peered":"2024-03-29T04:09:15.299361+0000","last_unstale":"2024-03-29T04:09:22.329606+0000","last_undegraded":"2024-03-29T04:09:22.329606+0000","last_fullsized":"2024-03-29T04:09:22.329606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656124+0000","last_change":"2024-03-29T04:09:16.322096+0000","last_active":"2024-03-29T04:09:22.656124+0000","last_peered":"2024-03-29T04:09:22.656124+0000","last_clean":"2024-03-29T04:09:22.656124+0000","last_became_active":"2024-03-29T04:09:16.321833+0000","last_became_peered":"2024-03-29T04:09:16.321833+0000","last_unstale":"2024-03-29T04:09:22.656124+0000","last_undegraded":"2024-03-29T04:09:22.656124+0000","last_fullsized":"2024-03-29T04:09:22.656124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656607+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.656607+0000","last_peered":"2024-03-29T04:09:22.656607+0000","last_clean":"2024-03-29T04:09:22.656607+0000","last_became_active":"2024-03-29T04:09:20.347088+0000","last_became_peered":"2024-03-29T04:09:20.347088+0000","last_unstale":"2024-03-29T04:09:22.656607+0000","last_undegraded":"2024-03-29T04:09:22.656607+0000","last_fullsized":"2024-03-29T04:09:22.656607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:45.089535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329185+0000","last_change":"2024-03-29T04:09:15.301823+0000","last_active":"2024-03-29T04:09:22.329185+0000","last_peered":"2024-03-29T04:09:22.329185+0000","last_clean":"2024-03-29T04:09:22.329185+0000","last_became_active":"2024-03-29T04:09:15.301182+0000","last_became_peered":"2024-03-29T04:09:15.301182+0000","last_unstale":"2024-03-29T04:09:22.329185+0000","last_undegraded":"2024-03-29T04:09:22.329185+0000","last_fullsized":"2024-03-29T04:09:22.329185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329886+0000","last_change":"2024-03-29T04:09:16.311797+0000","last_active":"2024-03-29T04:09:22.329886+0000","last_peered":"2024-03-29T04:09:22.329886+0000","last_clean":"2024-03-29T04:09:22.329886+0000","last_became_active":"2024-03-29T04:09:16.311544+0000","last_became_peered":"2024-03-29T04:09:16.311544+0000","last_unstale":"2024-03-29T04:09:22.329886+0000","last_undegraded":"2024-03-29T04:09:22.329886+0000","last_fullsized":"2024-03-29T04:09:22.329886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:41.184352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329592+0000","last_change":"2024-03-29T04:09:20.344647+0000","last_active":"2024-03-29T04:09:22.329592+0000","last_peered":"2024-03-29T04:09:22.329592+0000","last_clean":"2024-03-29T04:09:22.329592+0000","last_became_active":"2024-03-29T04:09:20.344278+0000","last_became_peered":"2024-03-29T04:09:20.344278+0000","last_unstale":"2024-03-29T04:09:22.329592+0000","last_undegraded":"2024-03-29T04:09:22.329592+0000","last_fullsized":"2024-03-29T04:09:22.329592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:15.627313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328975+0000","last_change":"2024-03-29T04:09:16.311872+0000","last_active":"2024-03-29T04:09:22.328975+0000","last_peered":"2024-03-29T04:09:22.328975+0000","last_clean":"2024-03-29T04:09:22.328975+0000","last_became_active":"2024-03-29T04:09:16.311576+0000","last_became_peered":"2024-03-29T04:09:16.311576+0000","last_unstale":"2024-03-29T04:09:22.328975+0000","last_undegraded":"2024-03-29T04:09:22.328975+0000","last_fullsized":"2024-03-29T04:09:22.328975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656255+0000","last_change":"2024-03-29T04:09:16.315268+0000","last_active":"2024-03-29T04:09:22.656255+0000","last_peered":"2024-03-29T04:09:22.656255+0000","last_clean":"2024-03-29T04:09:22.656255+0000","last_became_active":"2024-03-29T04:09:16.315002+0000","last_became_peered":"2024-03-29T04:09:16.315002+0000","last_unstale":"2024-03-29T04:09:22.656255+0000","last_undegraded":"2024-03-29T04:09:22.656255+0000","last_fullsized":"2024-03-29T04:09:22.656255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.323782+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.323504+0000","last_became_peered":"2024-03-29T04:09:16.323504+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656037+0000","last_change":"2024-03-29T04:09:16.316947+0000","last_active":"2024-03-29T04:09:22.656037+0000","last_peered":"2024-03-29T04:09:22.656037+0000","last_clean":"2024-03-29T04:09:22.656037+0000","last_became_active":"2024-03-29T04:09:16.316689+0000","last_became_peered":"2024-03-29T04:09:16.316689+0000","last_unstale":"2024-03-29T04:09:22.656037+0000","last_undegraded":"2024-03-29T04:09:22.656037+0000","last_fullsized":"2024-03-29T04:09:22.656037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329288+0000","last_change":"2024-03-29T04:09:16.319962+0000","last_active":"2024-03-29T04:09:22.329288+0000","last_peered":"2024-03-29T04:09:22.329288+0000","last_clean":"2024-03-29T04:09:22.329288+0000","last_became_active":"2024-03-29T04:09:16.319316+0000","last_became_peered":"2024-03-29T04:09:16.319316+0000","last_unstale":"2024-03-29T04:09:22.329288+0000","last_undegraded":"2024-03-29T04:09:22.329288+0000","last_fullsized":"2024-03-29T04:09:22.329288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655509+0000","last_change":"2024-03-29T04:09:16.312227+0000","last_active":"2024-03-29T04:09:22.655509+0000","last_peered":"2024-03-29T04:09:22.655509+0000","last_clean":"2024-03-29T04:09:22.655509+0000","last_became_active":"2024-03-29T04:09:16.311944+0000","last_became_peered":"2024-03-29T04:09:16.311944+0000","last_unstale":"2024-03-29T04:09:22.655509+0000","last_undegraded":"2024-03-29T04:09:22.655509+0000","last_fullsized":"2024-03-29T04:09:22.655509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330212+0000","last_change":"2024-03-29T04:09:16.306234+0000","last_active":"2024-03-29T04:09:22.330212+0000","last_peered":"2024-03-29T04:09:22.330212+0000","last_clean":"2024-03-29T04:09:22.330212+0000","last_became_active":"2024-03-29T04:09:16.305931+0000","last_became_peered":"2024-03-29T04:09:16.305931+0000","last_unstale":"2024-03-29T04:09:22.330212+0000","last_undegraded":"2024-03-29T04:09:22.330212+0000","last_fullsized":"2024-03-29T04:09:22.330212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329471+0000","last_change":"2024-03-29T04:09:16.310048+0000","last_active":"2024-03-29T04:09:22.329471+0000","last_peered":"2024-03-29T04:09:22.329471+0000","last_clean":"2024-03-29T04:09:22.329471+0000","last_became_active":"2024-03-29T04:09:16.309626+0000","last_became_peered":"2024-03-29T04:09:16.309626+0000","last_unstale":"2024-03-29T04:09:22.329471+0000","last_undegraded":"2024-03-29T04:09:22.329471+0000","last_fullsized":"2024-03-29T04:09:22.329471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654709+0000","last_change":"2024-03-29T04:09:16.324820+0000","last_active":"2024-03-29T04:09:22.654709+0000","last_peered":"2024-03-29T04:09:22.654709+0000","last_clean":"2024-03-29T04:09:22.654709+0000","last_became_active":"2024-03-29T04:09:16.324516+0000","last_became_peered":"2024-03-29T04:09:16.324516+0000","last_unstale":"2024-03-29T04:09:22.654709+0000","last_undegraded":"2024-03-29T04:09:22.654709+0000","last_fullsized":"2024-03-29T04:09:22.654709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329113+0000","last_change":"2024-03-29T04:09:16.318107+0000","last_active":"2024-03-29T04:09:22.329113+0000","last_peered":"2024-03-29T04:09:22.329113+0000","last_clean":"2024-03-29T04:09:22.329113+0000","last_became_active":"2024-03-29T04:09:16.317633+0000","last_became_peered":"2024-03-29T04:09:16.317633+0000","last_unstale":"2024-03-29T04:09:22.329113+0000","last_undegraded":"2024-03-29T04:09:22.329113+0000","last_fullsized":"2024-03-29T04:09:22.329113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654968+0000","last_change":"2024-03-29T04:09:16.317756+0000","last_active":"2024-03-29T04:09:22.654968+0000","last_peered":"2024-03-29T04:09:22.654968+0000","last_clean":"2024-03-29T04:09:22.654968+0000","last_became_active":"2024-03-29T04:09:16.317411+0000","last_became_peered":"2024-03-29T04:09:16.317411+0000","last_unstale":"2024-03-29T04:09:22.654968+0000","last_undegraded":"2024-03-29T04:09:22.654968+0000","last_fullsized":"2024-03-29T04:09:22.654968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655375+0000","last_change":"2024-03-29T04:09:16.314033+0000","last_active":"2024-03-29T04:09:22.655375+0000","last_peered":"2024-03-29T04:09:22.655375+0000","last_clean":"2024-03-29T04:09:22.655375+0000","last_became_active":"2024-03-29T04:09:16.313440+0000","last_became_peered":"2024-03-29T04:09:16.313440+0000","last_unstale":"2024-03-29T04:09:22.655375+0000","last_undegraded":"2024-03-29T04:09:22.655375+0000","last_fullsized":"2024-03-29T04:09:22.655375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329544+0000","last_change":"2024-03-29T04:09:16.318843+0000","last_active":"2024-03-29T04:09:22.329544+0000","last_peered":"2024-03-29T04:09:22.329544+0000","last_clean":"2024-03-29T04:09:22.329544+0000","last_became_active":"2024-03-29T04:09:16.318563+0000","last_became_peered":"2024-03-29T04:09:16.318563+0000","last_unstale":"2024-03-29T04:09:22.329544+0000","last_undegraded":"2024-03-29T04:09:22.329544+0000","last_fullsized":"2024-03-29T04:09:22.329544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656080+0000","last_change":"2024-03-29T04:09:16.313733+0000","last_active":"2024-03-29T04:09:22.656080+0000","last_peered":"2024-03-29T04:09:22.656080+0000","last_clean":"2024-03-29T04:09:22.656080+0000","last_became_active":"2024-03-29T04:09:16.313193+0000","last_became_peered":"2024-03-29T04:09:16.313193+0000","last_unstale":"2024-03-29T04:09:22.656080+0000","last_undegraded":"2024-03-29T04:09:22.656080+0000","last_fullsized":"2024-03-29T04:09:22.656080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:59.588666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655943+0000","last_change":"2024-03-29T04:09:16.300679+0000","last_active":"2024-03-29T04:09:22.655943+0000","last_peered":"2024-03-29T04:09:22.655943+0000","last_clean":"2024-03-29T04:09:22.655943+0000","last_became_active":"2024-03-29T04:09:16.300399+0000","last_became_peered":"2024-03-29T04:09:16.300399+0000","last_unstale":"2024-03-29T04:09:22.655943+0000","last_undegraded":"2024-03-29T04:09:22.655943+0000","last_fullsized":"2024-03-29T04:09:22.655943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:10.179988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655462+0000","last_change":"2024-03-29T04:09:16.314857+0000","last_active":"2024-03-29T04:09:22.655462+0000","last_peered":"2024-03-29T04:09:22.655462+0000","last_clean":"2024-03-29T04:09:22.655462+0000","last_became_active":"2024-03-29T04:09:16.314478+0000","last_became_peered":"2024-03-29T04:09:16.314478+0000","last_unstale":"2024-03-29T04:09:22.655462+0000","last_undegraded":"2024-03-29T04:09:22.655462+0000","last_fullsized":"2024-03-29T04:09:22.655462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654626+0000","last_change":"2024-03-29T04:09:16.314396+0000","last_active":"2024-03-29T04:09:22.654626+0000","last_peered":"2024-03-29T04:09:22.654626+0000","last_clean":"2024-03-29T04:09:22.654626+0000","last_became_active":"2024-03-29T04:09:16.313977+0000","last_became_peered":"2024-03-29T04:09:16.313977+0000","last_unstale":"2024-03-29T04:09:22.654626+0000","last_undegraded":"2024-03-29T04:09:22.654626+0000","last_fullsized":"2024-03-29T04:09:22.654626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655312+0000","last_change":"2024-03-29T04:09:16.320206+0000","last_active":"2024-03-29T04:09:22.655312+0000","last_peered":"2024-03-29T04:09:22.655312+0000","last_clean":"2024-03-29T04:09:22.655312+0000","last_became_active":"2024-03-29T04:09:16.319909+0000","last_became_peered":"2024-03-29T04:09:16.319909+0000","last_unstale":"2024-03-29T04:09:22.655312+0000","last_undegraded":"2024-03-29T04:09:22.655312+0000","last_fullsized":"2024-03-29T04:09:22.655312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:16.308762+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:16.308446+0000","last_became_peered":"2024-03-29T04:09:16.308446+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:27.406089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328992+0000","last_change":"2024-03-29T04:09:16.306843+0000","last_active":"2024-03-29T04:09:22.328992+0000","last_peered":"2024-03-29T04:09:22.328992+0000","last_clean":"2024-03-29T04:09:22.328992+0000","last_became_active":"2024-03-29T04:09:16.306402+0000","last_became_peered":"2024-03-29T04:09:16.306402+0000","last_unstale":"2024-03-29T04:09:22.328992+0000","last_undegraded":"2024-03-29T04:09:22.328992+0000","last_fullsized":"2024-03-29T04:09:22.328992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330172+0000","last_change":"2024-03-29T04:09:16.310846+0000","last_active":"2024-03-29T04:09:22.330172+0000","last_peered":"2024-03-29T04:09:22.330172+0000","last_clean":"2024-03-29T04:09:22.330172+0000","last_became_active":"2024-03-29T04:09:16.309820+0000","last_became_peered":"2024-03-29T04:09:16.309820+0000","last_unstale":"2024-03-29T04:09:22.330172+0000","last_undegraded":"2024-03-29T04:09:22.330172+0000","last_fullsized":"2024-03-29T04:09:22.330172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:18.601851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329788+0000","last_change":"2024-03-29T04:09:16.316170+0000","last_active":"2024-03-29T04:09:22.329788+0000","last_peered":"2024-03-29T04:09:22.329788+0000","last_clean":"2024-03-29T04:09:22.329788+0000","last_became_active":"2024-03-29T04:09:16.315896+0000","last_became_peered":"2024-03-29T04:09:16.315896+0000","last_unstale":"2024-03-29T04:09:22.329788+0000","last_undegraded":"2024-03-29T04:09:22.329788+0000","last_fullsized":"2024-03-29T04:09:22.329788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330150+0000","last_change":"2024-03-29T04:09:16.316842+0000","last_active":"2024-03-29T04:09:22.330150+0000","last_peered":"2024-03-29T04:09:22.330150+0000","last_clean":"2024-03-29T04:09:22.330150+0000","last_became_active":"2024-03-29T04:09:16.316573+0000","last_became_peered":"2024-03-29T04:09:16.316573+0000","last_unstale":"2024-03-29T04:09:22.330150+0000","last_undegraded":"2024-03-29T04:09:22.330150+0000","last_fullsized":"2024-03-29T04:09:22.330150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656222+0000","last_change":"2024-03-29T04:09:16.316209+0000","last_active":"2024-03-29T04:09:22.656222+0000","last_peered":"2024-03-29T04:09:22.656222+0000","last_clean":"2024-03-29T04:09:22.656222+0000","last_became_active":"2024-03-29T04:09:16.315942+0000","last_became_peered":"2024-03-29T04:09:16.315942+0000","last_unstale":"2024-03-29T04:09:22.656222+0000","last_undegraded":"2024-03-29T04:09:22.656222+0000","last_fullsized":"2024-03-29T04:09:22.656222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328767+0000","last_change":"2024-03-29T04:09:16.316731+0000","last_active":"2024-03-29T04:09:22.328767+0000","last_peered":"2024-03-29T04:09:22.328767+0000","last_clean":"2024-03-29T04:09:22.328767+0000","last_became_active":"2024-03-29T04:09:16.316418+0000","last_became_peered":"2024-03-29T04:09:16.316418+0000","last_unstale":"2024-03-29T04:09:22.328767+0000","last_undegraded":"2024-03-29T04:09:22.328767+0000","last_fullsized":"2024-03-29T04:09:22.328767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.312822+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.312495+0000","last_became_peered":"2024-03-29T04:09:16.312495+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654882+0000","last_change":"2024-03-29T04:09:16.316248+0000","last_active":"2024-03-29T04:09:22.654882+0000","last_peered":"2024-03-29T04:09:22.654882+0000","last_clean":"2024-03-29T04:09:22.654882+0000","last_became_active":"2024-03-29T04:09:16.315901+0000","last_became_peered":"2024-03-29T04:09:16.315901+0000","last_unstale":"2024-03-29T04:09:22.654882+0000","last_undegraded":"2024-03-29T04:09:22.654882+0000","last_fullsized":"2024-03-29T04:09:22.654882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655574+0000","last_change":"2024-03-29T04:09:16.307754+0000","last_active":"2024-03-29T04:09:22.655574+0000","last_peered":"2024-03-29T04:09:22.655574+0000","last_clean":"2024-03-29T04:09:22.655574+0000","last_became_active":"2024-03-29T04:09:16.307322+0000","last_became_peered":"2024-03-29T04:09:16.307322+0000","last_unstale":"2024-03-29T04:09:22.655574+0000","last_undegraded":"2024-03-29T04:09:22.655574+0000","last_fullsized":"2024-03-29T04:09:22.655574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.323875+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.323578+0000","last_became_peered":"2024-03-29T04:09:16.323578+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655497+0000","last_change":"2024-03-29T04:09:16.311321+0000","last_active":"2024-03-29T04:09:22.655497+0000","last_peered":"2024-03-29T04:09:22.655497+0000","last_clean":"2024-03-29T04:09:22.655497+0000","last_became_active":"2024-03-29T04:09:16.310990+0000","last_became_peered":"2024-03-29T04:09:16.310990+0000","last_unstale":"2024-03-29T04:09:22.655497+0000","last_undegraded":"2024-03-29T04:09:22.655497+0000","last_fullsized":"2024-03-29T04:09:22.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656870+0000","last_change":"2024-03-29T04:09:16.313513+0000","last_active":"2024-03-29T04:09:22.656870+0000","last_peered":"2024-03-29T04:09:22.656870+0000","last_clean":"2024-03-29T04:09:22.656870+0000","last_became_active":"2024-03-29T04:09:16.312390+0000","last_became_peered":"2024-03-29T04:09:16.312390+0000","last_unstale":"2024-03-29T04:09:22.656870+0000","last_undegraded":"2024-03-29T04:09:22.656870+0000","last_fullsized":"2024-03-29T04:09:22.656870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656162+0000","last_change":"2024-03-29T04:09:16.303994+0000","last_active":"2024-03-29T04:09:22.656162+0000","last_peered":"2024-03-29T04:09:22.656162+0000","last_clean":"2024-03-29T04:09:22.656162+0000","last_became_active":"2024-03-29T04:09:16.303670+0000","last_became_peered":"2024-03-29T04:09:16.303670+0000","last_unstale":"2024-03-29T04:09:22.656162+0000","last_undegraded":"2024-03-29T04:09:22.656162+0000","last_fullsized":"2024-03-29T04:09:22.656162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:33.656 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:09:33.656 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T04:09:33.656 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T04:09:33.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T04:09:33.657 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T04:09:33.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.812+0000 7f22e6ac4640 1 Processor -- start 2024-03-29T04:09:33.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.812+0000 7f22e6ac4640 1 -- start start 2024-03-29T04:09:33.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e6ac4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22e0162030 0x7f22e01624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e6ac4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e0141910 con 0x7f22e0141530 2024-03-29T04:09:33.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e6ac4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e0162af0 con 0x7f22e00b5240 2024-03-29T04:09:33.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e6ac4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f22e0162dd0 con 0x7f22e0162030 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22dffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22e0162030 0x7f22e01624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e4860640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f22e0141530 0x7f22e0071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57036/0 (socket says 172.21.15.181:57036) 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.813+0000 7f22e4860640 1 -- 172.21.15.181:0/4274821868 learned_addr learned my addr 172.21.15.181:0/4274821868 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22dffff640 1 -- 172.21.15.181:0/4274821868 >> v1:172.21.15.187:6789/0 conn(0x7f22e00b5240 legacy=0x7f22e0141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22df7fe640 1 -- 172.21.15.181:0/4274821868 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709359680 0 0) 0x7f22e0162af0 con 0x7f22e00b5240 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22dffff640 1 -- 172.21.15.181:0/4274821868 >> v1:172.21.15.181:6789/0 conn(0x7f22e0141530 legacy=0x7f22e0071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22dffff640 1 -- 172.21.15.181:0/4274821868 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e0071120 con 0x7f22e0162030 2024-03-29T04:09:33.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22dffff640 1 --2- 172.21.15.181:0/4274821868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e01624e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de75401a4db58bd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.814+0000 7f22df7fe640 1 -- 172.21.15.181:0/4274821868 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4037070 con 0x7f22e0162030 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22df7fe640 1 -- 172.21.15.181:0/4274821868 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d4030710 con 0x7f22e0162030 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22df7fe640 1 -- 172.21.15.181:0/4274821868 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4030ae0 con 0x7f22e0162030 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 -- 172.21.15.181:0/4274821868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 msgr2=0x7f22e01624e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/4274821868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e01624e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b710 comp rx=0 tx=0).stop 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 -- 172.21.15.181:0/4274821868 shutdown_connections 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/4274821868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e01624e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 -- 172.21.15.181:0/4274821868 >> 172.21.15.181:0/4274821868 conn(0x7f22e006f8b0 msgr2=0x7f22e006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 -- 172.21.15.181:0/4274821868 shutdown_connections 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.815+0000 7f22e6ac4640 1 -- 172.21.15.181:0/4274821868 wait complete. 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.816+0000 7f22e6ac4640 1 Processor -- start 2024-03-29T04:09:33.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.816+0000 7f22e6ac4640 1 -- start start 2024-03-29T04:09:33.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22e00b5240 0x7f22e00b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22e0141530 0x7f22e0144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0142320 con 0x7f22e0162030 2024-03-29T04:09:33.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22e00b1950 con 0x7f22e0141530 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e6ac4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22e01424a0 con 0x7f22e00b5240 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34374/0 (socket says 172.21.15.181:34374) 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 -- 172.21.15.181:0/3451600113 learned_addr learned my addr 172.21.15.181:0/3451600113 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e4860640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22e00b5240 0x7f22e00b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22dffff640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22e0141530 0x7f22e0144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 -- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22e00b5240 msgr2=0x7f22e00b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22e00b5240 0x7f22e00b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 -- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22e0141530 msgr2=0x7f22e0144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22e0141530 0x7f22e0144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.817+0000 7f22e5061640 1 -- 172.21.15.181:0/3451600113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e00b4a80 con 0x7f22e0162030 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22e5061640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c7e0 tx=0x7f22d800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8017070 con 0x7f22e0162030 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d8003dc0 con 0x7f22e0162030 2024-03-29T04:09:33.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d80052b0 con 0x7f22e0162030 2024-03-29T04:09:33.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e00b47a0 con 0x7f22e0162030 2024-03-29T04:09:33.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.818+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e0156170 con 0x7f22e0162030 2024-03-29T04:09:33.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.819+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e0160ac0 con 0x7f22e0162030 2024-03-29T04:09:33.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.819+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f22d801a070 con 0x7f22e0162030 2024-03-29T04:09:33.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.820+0000 7f22dd7fa640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 0x7f22cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:33.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.820+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f22d808b020 con 0x7f22e0162030 2024-03-29T04:09:33.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.820+0000 7f22e4860640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 0x7f22cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:33.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.820+0000 7f22e4860640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 0x7f22cc06e9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000f60 tx=0x7f22d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:33.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:33.824+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f22e0160ac0 con 0x7f22e0162030 2024-03-29T04:09:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.045+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f22e0075a10 con 0x7f22e0162030 2024-03-29T04:09:34.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.048+0000 7f22dd7fa640 1 -- 172.21.15.181:0/3451600113 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173742 (secure 0 0 0) 0x7f22d8058cb0 con 0x7f22e0162030 2024-03-29T04:09:34.030 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 msgr2=0x7f22cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 0x7f22cc06e9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000f60 tx=0x7f22d0009040 comp rx=0 tx=0).stop 2024-03-29T04:09:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 msgr2=0x7f22e0154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c7e0 tx=0x7f22d800cca0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 shutdown_connections 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22e0162030 0x7f22e0154af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22e0141530 0x7f22e0144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f22cc06c530 0x7f22cc06e9f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 --2- 172.21.15.181:0/3451600113 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22e00b5240 0x7f22e00b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.050+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 >> 172.21.15.181:0/3451600113 conn(0x7f22e006f8b0 msgr2=0x7f22e0160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.051+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 shutdown_connections 2024-03-29T04:09:34.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.051+0000 7f22e6ac4640 1 -- 172.21.15.181:0/3451600113 wait complete. 2024-03-29T04:09:34.041 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":4041148452},{"type":"v1","addr":"172.21.15.181:6833","nonce":4041148452}]},"active_addr":"172.21.15.181:6833/4041148452","active_change":"2024-03-29T04:08:55.565569+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1961155707}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2425085405}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3802204958}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2451592998}]}]} 2024-03-29T04:09:34.045 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T04:09:34.045 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T04:09:34.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:34.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 Processor -- start 2024-03-29T04:09:34.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 -- start start 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b815e950 0x7fe3b80d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 0x7fe3b80d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3b80d2f00 0x7fe3b80cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b80d3cb0 con 0x7fe3b80d2f00 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b80d4d00 con 0x7fe3b80d48c0 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.168+0000 7fe3bce4e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80d36f0 con 0x7fe3b815e950 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3b80d2f00 0x7fe3b80cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3affff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 0x7fe3b80d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3b80d2f00 0x7fe3b80cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34378/0 (socket says 172.21.15.181:34378) 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3affff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 0x7fe3b80d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53864/0 (socket says 172.21.15.181:53864) 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b815e950 0x7fe3b80d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/3254617643 learned_addr learned my addr 172.21.15.181:0/3254617643 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/3254617643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b815e950 msgr2=0x7fe3b80d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/3254617643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b815e950 0x7fe3b80d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/3254617643 >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 msgr2=0x7fe3b80d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/3254617643 >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 0x7fe3b80d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.169+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/3254617643 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b80d39d0 con 0x7fe3b80d2f00 2024-03-29T04:09:34.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/3254617643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b80d2f00 0x7fe3b80cef90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80736a0 tx=0x7fe3b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6474024fe6b908c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3b5d9b640 1 -- 172.21.15.181:0/3254617643 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b002cb00 con 0x7fe3b80d2f00 2024-03-29T04:09:34.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3b5d9b640 1 -- 172.21.15.181:0/3254617643 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b002cca0 con 0x7fe3b80d2f00 2024-03-29T04:09:34.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3b5d9b640 1 -- 172.21.15.181:0/3254617643 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0034610 con 0x7fe3b80d2f00 2024-03-29T04:09:34.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 -- 172.21.15.181:0/3254617643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b80d2f00 msgr2=0x7fe3b80cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/3254617643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b80d2f00 0x7fe3b80cef90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80736a0 tx=0x7fe3b002bf60 comp rx=0 tx=0).stop 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 -- 172.21.15.181:0/3254617643 shutdown_connections 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/3254617643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b80d2f00 0x7fe3b80cef90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/3254617643 >> v2:172.21.15.187:3300/0 conn(0x7fe3b80d48c0 0x7fe3b80d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/3254617643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b815e950 0x7fe3b80d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 -- 172.21.15.181:0/3254617643 >> 172.21.15.181:0/3254617643 conn(0x7fe3b80676e0 msgr2=0x7fe3b8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 -- 172.21.15.181:0/3254617643 shutdown_connections 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.170+0000 7fe3bce4e640 1 -- 172.21.15.181:0/3254617643 wait complete. 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.171+0000 7fe3bce4e640 1 Processor -- start 2024-03-29T04:09:34.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 -- start start 2024-03-29T04:09:34.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b80d2f00 0x7fe3b8158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3b80d48c0 0x7fe3b8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80d3390 con 0x7fe3b815e950 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b7da0 con 0x7fe3b80d48c0 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3bce4e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b815ed30 con 0x7fe3b80d2f00 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3b6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3b6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34394/0 (socket says 172.21.15.181:34394) 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/1489045332 learned_addr learned my addr 172.21.15.181:0/1489045332 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3affff640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3b80d48c0 0x7fe3b8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.172+0000 7fe3b659c640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b80d2f00 0x7fe3b8158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b80d2f00 msgr2=0x7fe3b8158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b80d2f00 0x7fe3b8158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3b80d48c0 msgr2=0x7fe3b8151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3b80d48c0 0x7fe3b8151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 -- 172.21.15.181:0/1489045332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b8158210 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3b6d9d640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0036040 tx=0x7fe3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0043020 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b002ce50 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b81584f0 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0009510 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.173+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b80cc6b0 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.174+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b8075a10 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.174+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe3b0046020 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.175+0000 7fe3aeffd640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 0x7fe39806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.175+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe3b00addb0 con 0x7fe3b815e950 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.175+0000 7fe3b659c640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 0x7fe39806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.156 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.175+0000 7fe3b659c640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 0x7fe39806e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009ac0 tx=0x7fe3a8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.158 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.179+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe3b004d030 con 0x7fe3b815e950 2024-03-29T04:09:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.354+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3b8152480 con 0x7fe3b815e950 2024-03-29T04:09:34.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.355+0000 7fe3aeffd640 1 -- 172.21.15.181:0/1489045332 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fe3b007bad0 con 0x7fe3b815e950 2024-03-29T04:09:34.336 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:34.336 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.357+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 msgr2=0x7fe39806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.357+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 0x7fe39806e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009ac0 tx=0x7fe3a8009190 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.357+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 msgr2=0x7fe3b815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.357+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0036040 tx=0x7fe3b0004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 shutdown_connections 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3b815e950 0x7fe3b815b080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe3b80d48c0 0x7fe3b8151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe39806c530 0x7fe39806e9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 --2- 172.21.15.181:0/1489045332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe3b80d2f00 0x7fe3b8158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 >> 172.21.15.181:0/1489045332 conn(0x7fe3b80676e0 msgr2=0x7fe3b80d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 shutdown_connections 2024-03-29T04:09:34.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.358+0000 7fe3bce4e640 1 -- 172.21.15.181:0/1489045332 wait complete. 2024-03-29T04:09:34.349 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T04:09:34.349 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:34.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.469+0000 7f082cb0e640 1 Processor -- start 2024-03-29T04:09:34.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.469+0000 7f082cb0e640 1 -- start start 2024-03-29T04:09:34.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082cb0e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 0x7f0828141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082cb0e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f082813f640 0x7f0828142310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082cb0e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0828161040 con 0x7f082813f260 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082cb0e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f082815db70 con 0x7f082813f640 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082cb0e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0828142b90 con 0x7f082806bc90 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f0826d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f082813f640 0x7f0828142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f0825d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 0x7f0828141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082659c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f082813f260 0x7f082806b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:34988/0 (socket says 172.21.15.181:34988) 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f082659c640 1 -- 172.21.15.181:0/698382999 learned_addr learned my addr 172.21.15.181:0/698382999 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.470+0000 7f0826d9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f082813f640 0x7f0828142310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53890/0 (socket says 172.21.15.181:53890) 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.471+0000 7f0826d9d640 1 -- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 msgr2=0x7f0828141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.471+0000 7f0826d9d640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 0x7f0828141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.471+0000 7f0826d9d640 1 -- 172.21.15.181:0/698382999 >> v1:172.21.15.187:6789/0 conn(0x7f082813f260 legacy=0x7f082806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.471+0000 7f0826d9d640 1 -- 172.21.15.181:0/698382999 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828160280 con 0x7f082813f640 2024-03-29T04:09:34.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.471+0000 7f0825d9b640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 0x7f0828141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f0826d9d640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f640 0x7f0828142310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08200097a0 tx=0x7f082002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75a7d552e4e7904 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082559a640 1 -- 172.21.15.181:0/698382999 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200315a0 con 0x7f082813f640 2024-03-29T04:09:34.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082559a640 1 -- 172.21.15.181:0/698382999 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0820037070 con 0x7f082813f640 2024-03-29T04:09:34.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082559a640 1 -- 172.21.15.181:0/698382999 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820031960 con 0x7f082813f640 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 -- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f640 msgr2=0x7f0828142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f640 0x7f0828142310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08200097a0 tx=0x7f082002b840 comp rx=0 tx=0).stop 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 -- 172.21.15.181:0/698382999 shutdown_connections 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f640 0x7f0828142310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 --2- 172.21.15.181:0/698382999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082806bc90 0x7f0828141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 -- 172.21.15.181:0/698382999 >> 172.21.15.181:0/698382999 conn(0x7f082806c690 msgr2=0x7f082806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 -- 172.21.15.181:0/698382999 shutdown_connections 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.472+0000 7f082cb0e640 1 -- 172.21.15.181:0/698382999 wait complete. 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 Processor -- start 2024-03-29T04:09:34.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 -- start start 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 0x7f08281f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082813f640 0x7f0828155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0828160b20 con 0x7f082806bc90 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0828160840 con 0x7f082813f260 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082cb0e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0828142850 con 0x7f082813f640 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.474+0000 7f082659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f082659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34404/0 (socket says 172.21.15.181:34404) 2024-03-29T04:09:34.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f082659c640 1 -- 172.21.15.181:0/3143310311 learned_addr learned my addr 172.21.15.181:0/3143310311 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 0x7f08281f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0826d9d640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082813f640 0x7f0828155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 -- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082813f640 msgr2=0x7f0828155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082813f640 0x7f0828155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 -- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 msgr2=0x7f08281f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 -- 172.21.15.181:0/3143310311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08281f6db0 con 0x7f082813f260 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f082659c640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0825d9b640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 0x7f08281f6870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f081c00e930 tx=0x7f081c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c017070 con 0x7f082813f260 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c00ca00 con 0x7f082813f260 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082813d750 con 0x7f082813f260 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c00cdd0 con 0x7f082813f260 2024-03-29T04:09:34.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.475+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082813dd10 con 0x7f082813f260 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.476+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0828075a10 con 0x7f082813f260 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.477+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f081c01a070 con 0x7f082813f260 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.477+0000 7f0816ffd640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 0x7f080806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.477+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f081c08b7e0 con 0x7f082813f260 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.477+0000 7f082659c640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 0x7f080806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.477+0000 7f082659c640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 0x7f080806ea20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08280d1e30 tx=0x7f0818009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.482+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f08281f6db0 con 0x7f082813f260 2024-03-29T04:09:34.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.657+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0828141d00 con 0x7f082813f260 2024-03-29T04:09:34.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.658+0000 7f0816ffd640 1 -- 172.21.15.181:0/3143310311 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f081c059480 con 0x7f082813f260 2024-03-29T04:09:34.638 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:34.638 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:34.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 msgr2=0x7f080806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 0x7f080806ea20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08280d1e30 tx=0x7f0818009040 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 msgr2=0x7f08281f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 0x7f08281f6870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f081c00e930 tx=0x7f081c00edf0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 shutdown_connections 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f080806c560 0x7f080806ea20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082813f640 0x7f0828155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082813f260 0x7f08281f6870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 --2- 172.21.15.181:0/3143310311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f082806bc90 0x7f08281f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.660+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 >> 172.21.15.181:0/3143310311 conn(0x7f082806c690 msgr2=0x7f08281532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.661+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 shutdown_connections 2024-03-29T04:09:34.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.661+0000 7f082cb0e640 1 -- 172.21.15.181:0/3143310311 wait complete. 2024-03-29T04:09:34.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T04:09:34.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T04:09:34.653 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T04:09:34.653 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T04:09:34.653 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T04:09:34.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T04:09:34.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T04:09:34.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T04:09:34.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.843+0000 7f3dcb332640 1 Processor -- start 2024-03-29T04:09:34.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.843+0000 7f3dcb332640 1 -- start start 2024-03-29T04:09:34.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.844+0000 7f3dcb332640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 0x7f3dc4072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.844+0000 7f3dcb332640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc40b4290 con 0x7f3dc4162850 2024-03-29T04:09:34.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.844+0000 7f3dcb332640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc40b4550 con 0x7f3dc40b3e00 2024-03-29T04:09:34.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.844+0000 7f3dcb332640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc40b4830 con 0x7f3dc406bc90 2024-03-29T04:09:34.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7fb570d6d640 1 Processor -- start 2024-03-29T04:09:34.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7fb570d6d640 1 -- start start 2024-03-29T04:09:34.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7efd79f24640 1 Processor -- start 2024-03-29T04:09:34.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7efd79f24640 1 -- start start 2024-03-29T04:09:34.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7f3dc90ce640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3dc4162850 0x7f3dc406b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57050/0 (socket says 172.21.15.181:57050) 2024-03-29T04:09:34.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7f3dc90ce640 1 -- 172.21.15.181:0/2770010063 learned_addr learned my addr 172.21.15.181:0/2770010063 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:34.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.845+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/2770010063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 0x7f3dc4072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7f3dc88cd640 1 -- 172.21.15.181:0/2770010063 >> v1:172.21.15.187:6789/0 conn(0x7f3dc40b3e00 legacy=0x7f3dc406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7f3dc88cd640 1 -- 172.21.15.181:0/2770010063 >> v1:172.21.15.181:6789/0 conn(0x7f3dc4162850 legacy=0x7f3dc406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7f3dc88cd640 1 -- 172.21.15.181:0/2770010063 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc406e450 con 0x7f3dc406bc90 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7f9b85740640 1 Processor -- start 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7f9b85740640 1 -- start start 2024-03-29T04:09:34.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7efd79f24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7fb570d6d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5640543b0 0x7fb564037c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7efd79f24640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6c056660 con 0x7efd6c03a230 2024-03-29T04:09:34.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7efd79f24640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6c0553a0 con 0x7efd6c00a200 2024-03-29T04:09:34.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7efd79f24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efd6c055c90 con 0x7efd6c009e20 2024-03-29T04:09:34.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7fb570d6d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 0x7fb564052c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51100/0 (socket says 172.21.15.181:51100) 2024-03-29T04:09:34.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 -- 172.21.15.181:0/2949905287 learned_addr learned my addr 172.21.15.181:0/2949905287 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7fb570d6d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb564054800 con 0x7fb564033980 2024-03-29T04:09:34.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7fb570d6d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb564053df0 con 0x7fb564038170 2024-03-29T04:09:34.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.846+0000 7fb570d6d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb5640540d0 con 0x7fb5640543b0 2024-03-29T04:09:34.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5640543b0 0x7fb564037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb5640543b0 0x7fb564037c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53908/0 (socket says 172.21.15.181:53908) 2024-03-29T04:09:34.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb56a59c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb564033980 0x7fb564007180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35004/0 (socket says 172.21.15.181:35004) 2024-03-29T04:09:34.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 -- 172.21.15.181:0/2184120009 learned_addr learned my addr 172.21.15.181:0/2184120009 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/2770010063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 0x7f3dc4072ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009dc0 tx=0x7f3dbc02bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abfce2f01f55c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7f9b85740640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b78005e80 0x7f9b780062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb56ad9d640 1 --2- 172.21.15.181:0/2184120009 >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 0x7fb564052c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 -- 172.21.15.181:0/2184120009 >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 msgr2=0x7fb564052c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7f9b7effd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9b780eacf0 0x7f9b7804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35022/0 (socket says 172.21.15.181:35022) 2024-03-29T04:09:34.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb56959a640 1 -- 172.21.15.181:0/2184120009 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551623968 0 0) 0x7fb564054800 con 0x7fb564033980 2024-03-29T04:09:34.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7f9b7effd640 1 -- 172.21.15.181:0/4023627531 learned_addr learned my addr 172.21.15.181:0/4023627531 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:34.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7f9b7e7fc640 1 --2- 172.21.15.181:0/4023627531 >> v2:172.21.15.187:3300/0 conn(0x7f9b78005e80 0x7f9b780062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 --2- 172.21.15.181:0/2184120009 >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 0x7fb564052c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 -- 172.21.15.181:0/2184120009 >> v1:172.21.15.187:6789/0 conn(0x7fb564033980 legacy=0x7fb564007180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7fb569d9b640 1 -- 172.21.15.181:0/2184120009 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb564006c60 con 0x7fb5640543b0 2024-03-29T04:09:34.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 -- 172.21.15.181:0/2949905287 >> v1:172.21.15.187:6789/0 conn(0x7efd6c00a200 legacy=0x7efd6c051cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b78053710 con 0x7f9b78006800 2024-03-29T04:09:34.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 -- 172.21.15.181:0/2949905287 >> v1:172.21.15.181:6789/0 conn(0x7efd6c03a230 legacy=0x7efd6c03a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b780539f0 con 0x7f9b780eacf0 2024-03-29T04:09:34.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.847+0000 7efd737fe640 1 -- 172.21.15.181:0/2949905287 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c055640 con 0x7efd6c009e20 2024-03-29T04:09:34.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b78054320 con 0x7f9b78005e80 2024-03-29T04:09:34.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7efd737fe640 1 --2- 172.21.15.181:0/2949905287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd68009880 tx=0x7efd6802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7da481e1803a092 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f3dbbfff640 1 -- 172.21.15.181:0/2770010063 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc02d8f0 con 0x7f3dc406bc90 2024-03-29T04:09:34.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7fb56ad9d640 1 --2- 172.21.15.181:0/2184120009 >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 0x7fb564052c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:34.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f629a4c0640 1 Processor -- start 2024-03-29T04:09:34.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f3dbbfff640 1 -- 172.21.15.181:0/2770010063 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc02da90 con 0x7f3dc406bc90 2024-03-29T04:09:34.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7fe932f4d640 1 Processor -- start 2024-03-29T04:09:34.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dbbfff640 1 -- 172.21.15.181:0/2770010063 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc02de60 con 0x7f3dc406bc90 2024-03-29T04:09:34.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b7e7fc640 1 -- 172.21.15.181:0/4023627531 >> v1:172.21.15.187:6789/0 conn(0x7f9b780eacf0 legacy=0x7f9b7804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b7e7fc640 1 -- 172.21.15.181:0/4023627531 >> v1:172.21.15.181:6789/0 conn(0x7f9b78006800 legacy=0x7f9b78006c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b7dffb640 1 -- 172.21.15.181:0/4023627531 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210178043 0 0) 0x7f9b780539f0 con 0x7f9b780eacf0 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.848+0000 7f9b7e7fc640 1 -- 172.21.15.181:0/4023627531 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b78053cd0 con 0x7f9b78005e80 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7efd727fc640 1 -- 172.21.15.181:0/2949905287 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6802d040 con 0x7efd6c009e20 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7efd727fc640 1 -- 172.21.15.181:0/2949905287 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68004230 con 0x7efd6c009e20 2024-03-29T04:09:34.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dcb332640 1 -- 172.21.15.181:0/2770010063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 msgr2=0x7f3dc4072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7fb569d9b640 1 --2- 172.21.15.181:0/2184120009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5640543b0 0x7fb564037c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb56c06a400 tx=0x7fb56c069360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6bd43f30c7ebc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dcb332640 1 --2- 172.21.15.181:0/2770010063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 0x7f3dc4072ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009dc0 tx=0x7f3dbc02bbb0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dcb332640 1 -- 172.21.15.181:0/2770010063 shutdown_connections 2024-03-29T04:09:34.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dcb332640 1 --2- 172.21.15.181:0/2770010063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc406bc90 0x7f3dc4072ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f3dcb332640 1 -- 172.21.15.181:0/2770010063 >> 172.21.15.181:0/2770010063 conn(0x7f3dc40b2b00 msgr2=0x7f3dc40b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7efd727fc640 1 -- 172.21.15.181:0/2949905287 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68003610 con 0x7efd6c009e20 2024-03-29T04:09:34.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fe932f4d640 1 -- start start 2024-03-29T04:09:34.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.849+0000 7f629a4c0640 1 -- start start 2024-03-29T04:09:34.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 -- 172.21.15.181:0/2949905287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 msgr2=0x7efd6c056120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 --2- 172.21.15.181:0/2949905287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd68009880 tx=0x7efd6802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fb56959a640 1 -- 172.21.15.181:0/2184120009 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0a3070 con 0x7fb5640543b0 2024-03-29T04:09:34.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fb56959a640 1 -- 172.21.15.181:0/2184120009 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c09e640 con 0x7fb5640543b0 2024-03-29T04:09:34.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fb56959a640 1 -- 172.21.15.181:0/2184120009 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c09eab0 con 0x7fb5640543b0 2024-03-29T04:09:34.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 -- 172.21.15.181:0/2949905287 shutdown_connections 2024-03-29T04:09:34.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 --2- 172.21.15.181:0/2949905287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c056120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 -- 172.21.15.181:0/2949905287 >> 172.21.15.181:0/2949905287 conn(0x7efd6c0537a0 msgr2=0x7efd6c053b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f9b7e7fc640 1 --2- 172.21.15.181:0/4023627531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78005e80 0x7f9b780062c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097a0 tx=0x7f9b7402b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=730ae5535481c2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f3dcb332640 1 -- 172.21.15.181:0/2770010063 shutdown_connections 2024-03-29T04:09:34.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f3dcb332640 1 -- 172.21.15.181:0/2770010063 wait complete. 2024-03-29T04:09:34.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f629a4c0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f629415a5c0 0x7f629415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fa269800640 1 Processor -- start 2024-03-29T04:09:34.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f629a4c0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f629415e880 con 0x7f62940b22f0 2024-03-29T04:09:34.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7fa269800640 1 -- start start 2024-03-29T04:09:34.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f629a4c0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f629415eb60 con 0x7f62940cf860 2024-03-29T04:09:34.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f629a4c0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f629415ee40 con 0x7f629415a5c0 2024-03-29T04:09:34.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f9b7dffb640 1 -- 172.21.15.181:0/4023627531 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74039070 con 0x7f9b78005e80 2024-03-29T04:09:34.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7efd79f24640 1 -- 172.21.15.181:0/2949905287 shutdown_connections 2024-03-29T04:09:34.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7efd79f24640 1 -- 172.21.15.181:0/2949905287 wait complete. 2024-03-29T04:09:34.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f6299cbf640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f62940cf860 0x7f62940ccae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35030/0 (socket says 172.21.15.181:35030) 2024-03-29T04:09:34.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f6299cbf640 1 -- 172.21.15.181:0/1241790416 learned_addr learned my addr 172.21.15.181:0/1241790416 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:34.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 -- 172.21.15.181:0/2184120009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5640543b0 msgr2=0x7fb564037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 --2- 172.21.15.181:0/2184120009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5640543b0 0x7fb564037c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb56c06a400 tx=0x7fb56c069360 comp rx=0 tx=0).stop 2024-03-29T04:09:34.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f6298cbd640 1 --2- 172.21.15.181:0/1241790416 >> v2:172.21.15.181:3300/0 conn(0x7f629415a5c0 0x7f629415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe932f4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe92c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe932f4d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c0ca8f0 con 0x7fe92c0b2390 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe932f4d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92c06f250 con 0x7fe92c06e450 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe932f4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c0cc6f0 con 0x7fe92c15e450 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 -- 172.21.15.181:0/2184120009 shutdown_connections 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 --2- 172.21.15.181:0/2184120009 >> v2:172.21.15.181:3300/0 conn(0x7fb564038170 0x7fb564052c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 --2- 172.21.15.181:0/2184120009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb5640543b0 0x7fb564037c30 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb56c06a400 tx=0x7fb56c069360 comp rx=0 tx=0).stop 2024-03-29T04:09:34.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fb570d6d640 1 -- 172.21.15.181:0/2184120009 >> 172.21.15.181:0/2184120009 conn(0x7fb564009fa0 msgr2=0x7fb56400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe93174a640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe92c06e450 0x7fe92c0b1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35042/0 (socket says 172.21.15.181:35042) 2024-03-29T04:09:34.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe93174a640 1 -- 172.21.15.181:0/2716572270 learned_addr learned my addr 172.21.15.181:0/2716572270 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:34.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f9b7dffb640 1 -- 172.21.15.181:0/4023627531 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b740316f0 con 0x7f9b78005e80 2024-03-29T04:09:34.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.850+0000 7f9b7dffb640 1 -- 172.21.15.181:0/4023627531 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74031ac0 con 0x7f9b78005e80 2024-03-29T04:09:34.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665987335 0 0) 0x7f629415eb60 con 0x7f62940cf860 2024-03-29T04:09:34.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6280007930 con 0x7f62940cf860 2024-03-29T04:09:34.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe92c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fa269800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa26406c260 0x7fa26406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276433580 0 0) 0x7fe92c06f250 con 0x7fe92c06e450 2024-03-29T04:09:34.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe918003520 con 0x7fe92c06e450 2024-03-29T04:09:34.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa269800640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2640d1c50 con 0x7fa26406bac0 2024-03-29T04:09:34.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f3dcb332640 1 Processor -- start 2024-03-29T04:09:34.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa269800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2640b3fe0 con 0x7fa264162970 2024-03-29T04:09:34.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa269800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa2640d1a00 con 0x7fa26406c260 2024-03-29T04:09:34.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1426564575 0 0) 0x7f6280007930 con 0x7f62940cf860 2024-03-29T04:09:34.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.851+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 >> v2:172.21.15.181:3300/0 conn(0x7f629415a5c0 msgr2=0x7f629415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa26406c260 0x7fa26406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa26406c260 0x7fa26406b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53928/0 (socket says 172.21.15.181:53928) 2024-03-29T04:09:34.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78005e80 msgr2=0x7f9b780062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3543295294 0 0) 0x7fe918003520 con 0x7fe92c06e450 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 msgr2=0x7fe92c06ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe92c06ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 >> v1:172.21.15.181:6789/0 conn(0x7fe92c0b2390 legacy=0x7fe92c0ce3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0cab60 con 0x7fe92c06e450 2024-03-29T04:09:34.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f9b85740640 1 --2- 172.21.15.181:0/4023627531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78005e80 0x7f9b780062c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097a0 tx=0x7f9b7402b810 comp rx=0 tx=0).stop 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fe9240019f0 con 0x7fe92c06e450 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe92c06ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 >> v1:172.21.15.187:6789/0 conn(0x7fe92c06e450 legacy=0x7fe92c0b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9180039b0 0x7fe918003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9180043d0 0x7fe9180048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe9180050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c0cc6f0 con 0x7fe9180043d0 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe91800b9c0 con 0x7fe9180039b0 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe91800bbc0 con 0x7fe92c15e450 2024-03-29T04:09:34.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9180039b0 0x7fe918003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 -- 172.21.15.181:0/4123980067 learned_addr learned my addr 172.21.15.181:0/4123980067 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 shutdown_connections 2024-03-29T04:09:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f9b85740640 1 --2- 172.21.15.181:0/4023627531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b78005e80 0x7f9b780062c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 >> 172.21.15.181:0/4023627531 conn(0x7f9b78009fa0 msgr2=0x7f9b7800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 --2- 172.21.15.181:0/1241790416 >> v2:172.21.15.181:3300/0 conn(0x7f629415a5c0 0x7f629415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 >> v1:172.21.15.181:6789/0 conn(0x7f62940b22f0 legacy=0x7f629415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629415aac0 con 0x7f62940cf860 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f628c0019f0 con 0x7f62940cf860 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 >> v1:172.21.15.187:6789/0 conn(0x7f62940cf860 legacy=0x7f62940ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6280007dc0 0x7f62800081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280008c10 0x7f6280009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62800145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f629415ee40 con 0x7f6280008c10 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6280007700 con 0x7f6280007dc0 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6280014b00 con 0x7f62800095d0 2024-03-29T04:09:34.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f6299cbf640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62800145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 -- 172.21.15.181:0/4123980067 >> v1:172.21.15.187:6789/0 conn(0x7fa264162970 legacy=0x7fa2640add70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 -- 172.21.15.181:0/4123980067 >> v1:172.21.15.181:6789/0 conn(0x7fa26406bac0 legacy=0x7fa2640b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fe93174a640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9180043d0 0x7fe9180048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe93274c640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe9180050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 shutdown_connections 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f9b85740640 1 -- 172.21.15.181:0/4023627531 wait complete. 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7efd79f24640 1 Processor -- start 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7efd79f24640 1 -- start start 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f6298cbd640 1 --2- 172.21.15.181:0/1241790416 >> v2:172.21.15.181:3300/0 conn(0x7f629415a5c0 0x7f629415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7f3dcb332640 1 -- start start 2024-03-29T04:09:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.852+0000 7fa2637fe640 1 -- 172.21.15.181:0/4123980067 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2640b42c0 con 0x7fa26406c260 2024-03-29T04:09:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6298cbd640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280008c10 0x7f6280009090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f3dcb332640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 0x7f3dc414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe931f4b640 1 -- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 msgr2=0x7fe9180050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c15e450 0x7fe9180050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe931f4b640 1 -- 172.21.15.181:0/2716572270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9180043d0 msgr2=0x7fe9180048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f3dcb332640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc4162850 0x7f3dc41648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9180043d0 0x7fe9180048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe931f4b640 1 -- 172.21.15.181:0/2716572270 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c0cab60 con 0x7fe9180039b0 2024-03-29T04:09:34.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fe93174a640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9180043d0 0x7fe9180048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f62994be640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6280007dc0 0x7f62800081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6299cbf640 1 -- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6280007dc0 msgr2=0x7f62800081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6299cbf640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6280007dc0 0x7f62800081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6299cbf640 1 -- 172.21.15.181:0/1241790416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280008c10 msgr2=0x7f6280009090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6299cbf640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280008c10 0x7f6280009090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6299cbf640 1 -- 172.21.15.181:0/1241790416 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629415aac0 con 0x7f62800095d0 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7f6298cbd640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280008c10 0x7f6280009090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dcb332640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc414ee90 0x7f3dc4162c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dcb332640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc406f640 con 0x7f3dc4162850 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dcb332640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc406ea10 con 0x7f3dc414ee90 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dcb332640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc406f7c0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 0x7f3dc414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 0x7f3dc414ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51140/0 (socket says 172.21.15.181:51140) 2024-03-29T04:09:34.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 -- 172.21.15.181:0/3590196233 learned_addr learned my addr 172.21.15.181:0/3590196233 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.853+0000 7fb570d6d640 1 -- 172.21.15.181:0/2184120009 shutdown_connections 2024-03-29T04:09:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc4162850 0x7f3dc41648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7fb570d6d640 1 -- 172.21.15.181:0/2184120009 wait complete. 2024-03-29T04:09:34.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9180039b0 0x7fe918003db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009cc0 tx=0x7fe918005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ff2f1ad2e23a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c187190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6c00a200 0x7efd6c178da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd6c03a230 0x7efd6c04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c00a660 con 0x7efd6c00a200 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c052790 con 0x7efd6c03a230 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd79f24640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efd6c052400 con 0x7efd6c009e20 2024-03-29T04:09:34.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc98cf640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc414ee90 0x7f3dc4162c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 -- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc414ee90 msgr2=0x7f3dc4162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dc414ee90 0x7f3dc4162c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 -- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc4162850 msgr2=0x7f3dc41648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc4162850 0x7f3dc41648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc90ce640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc414b510 con 0x7f3dc40b3e00 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c187190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd73fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd6c03a230 0x7efd6c04fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd73fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd6c03a230 0x7efd6c04fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53950/0 (socket says 172.21.15.181:53950) 2024-03-29T04:09:34.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc4162850 0x7f3dc41648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f3dc90ce640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 0x7f3dc414ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000c750 tx=0x7f3dc000cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f6299cbf640 1 -- 172.21.15.181:0/1241790416 reap_dead start 2024-03-29T04:09:34.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c039070 con 0x7fe9180039b0 2024-03-29T04:09:34.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c02c920 con 0x7fe9180039b0 2024-03-29T04:09:34.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7fe930f49640 1 -- 172.21.15.181:0/2716572270 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c009860 con 0x7fe9180039b0 2024-03-29T04:09:34.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f9b85740640 1 Processor -- start 2024-03-29T04:09:34.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.854+0000 7efd73fff640 1 -- 172.21.15.181:0/1586521859 learned_addr learned my addr 172.21.15.181:0/1586521859 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f9b85740640 1 -- start start 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd72ffd640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6c00a200 0x7efd6c178da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 -- 172.21.15.181:0/1586521859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd6c03a230 msgr2=0x7efd6c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd6c03a230 0x7efd6c04fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 -- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6c00a200 msgr2=0x7efd6c178da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6c00a200 0x7efd6c178da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1792e0 con 0x7efd6c009e20 2024-03-29T04:09:34.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f6299cbf640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62800145c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f628c01b990 tx=0x7f6280009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d39cd72472e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7efd737fe640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c187190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd68004000 tx=0x7efd68004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0018070 con 0x7f3dc40b3e00 2024-03-29T04:09:34.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00072f0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00047e0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c048070 con 0x7f62800095d0 2024-03-29T04:09:34.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c043630 con 0x7f62800095d0 2024-03-29T04:09:34.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f628bfff640 1 -- 172.21.15.181:0/1241790416 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284009840 con 0x7f62800095d0 2024-03-29T04:09:34.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.855+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc414b7f0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fe932f4d640 1 -- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9180039b0 msgr2=0x7fe918003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fe932f4d640 1 --2- 172.21.15.181:0/2716572270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9180039b0 0x7fe918003db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009cc0 tx=0x7fe918005830 comp rx=0 tx=0).stop 2024-03-29T04:09:34.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fb570d6d640 1 Processor -- start 2024-03-29T04:09:34.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc414bdb0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fe93274c640 1 -- 172.21.15.181:0/2716572270 reap_dead start 2024-03-29T04:09:34.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fe932f4d640 1 -- 172.21.15.181:0/2716572270 shutdown_connections 2024-03-29T04:09:34.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fe932f4d640 1 -- 172.21.15.181:0/2716572270 >> 172.21.15.181:0/2716572270 conn(0x7fe92c0ccc80 msgr2=0x7fe92c0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fb570d6d640 1 -- start start 2024-03-29T04:09:34.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68005b90 con 0x7efd6c009e20 2024-03-29T04:09:34.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68005d30 con 0x7efd6c009e20 2024-03-29T04:09:34.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7f9b85740640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b78005e80 0x7f9b780df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7f9b85740640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b78006800 0x7f9b78036d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7f629a4c0640 1 -- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 msgr2=0x7f62800145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7f629a4c0640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62800145c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f628c01b990 tx=0x7f6280009c40 comp rx=0 tx=0).stop 2024-03-29T04:09:34.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.856+0000 7fa2637fe640 1 --2- 172.21.15.181:0/4123980067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa26406c260 0x7fa26406b580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2640d1b80 tx=0x7fa25802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de19fa467e97e030 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68031850 con 0x7efd6c009e20 2024-03-29T04:09:34.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c04dfb0 con 0x7efd6c009e20 2024-03-29T04:09:34.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fe932f4d640 1 -- 172.21.15.181:0/2716572270 shutdown_connections 2024-03-29T04:09:34.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c04e570 con 0x7efd6c009e20 2024-03-29T04:09:34.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fa262ffd640 1 -- 172.21.15.181:0/4123980067 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258035020 con 0x7fa26406c260 2024-03-29T04:09:34.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fa262ffd640 1 -- 172.21.15.181:0/4123980067 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258031640 con 0x7fa26406c260 2024-03-29T04:09:34.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f629a4c0640 1 -- 172.21.15.181:0/1241790416 shutdown_connections 2024-03-29T04:09:34.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f629a4c0640 1 --2- 172.21.15.181:0/1241790416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62800145c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f629a4c0640 1 -- 172.21.15.181:0/1241790416 >> 172.21.15.181:0/1241790416 conn(0x7f62940b1800 msgr2=0x7f62940b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fe932f4d640 1 -- 172.21.15.181:0/2716572270 wait complete. 2024-03-29T04:09:34.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f629a4c0640 1 -- 172.21.15.181:0/1241790416 shutdown_connections 2024-03-29T04:09:34.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fb570d6d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb564033980 0x7fb5640344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7fb570d6d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 0x7fb564036510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dc414bff0 con 0x7f3dc40b3e00 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb569d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 0x7fb564036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb570d6d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 0x7fb5640490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb569d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 0x7fb564036510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34448/0 (socket says 172.21.15.181:34448) 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb56ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 0x7fb5640490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb56ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 0x7fb5640490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51172/0 (socket says 172.21.15.181:51172) 2024-03-29T04:09:34.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa262ffd640 1 -- 172.21.15.181:0/4123980067 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258035020 con 0x7fa26406c260 2024-03-29T04:09:34.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb56ad9d640 1 -- 172.21.15.181:0/3868488495 learned_addr learned my addr 172.21.15.181:0/3868488495 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa269800640 1 -- 172.21.15.181:0/4123980067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa26406c260 msgr2=0x7fa26406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb564038710 con 0x7fb564038170 2024-03-29T04:09:34.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb564038590 con 0x7fb564033980 2024-03-29T04:09:34.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb564053180 con 0x7fb564033060 2024-03-29T04:09:34.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa269800640 1 --2- 172.21.15.181:0/4123980067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa26406c260 0x7fa26406b580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2640d1b80 tx=0x7fa25802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fb56a59c640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb564033980 0x7fb5640344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa269800640 1 -- 172.21.15.181:0/4123980067 shutdown_connections 2024-03-29T04:09:34.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa269800640 1 --2- 172.21.15.181:0/4123980067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa26406c260 0x7fa26406b580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b78005e80 0x7f9b780df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b7effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b78005e80 0x7f9b780df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51156/0 (socket says 172.21.15.181:51156) 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b7effd640 1 -- 172.21.15.181:0/3527032522 learned_addr learned my addr 172.21.15.181:0/3527032522 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b85740640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b780eacf0 0x7f9b78044ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b780547b0 con 0x7f9b78006800 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78050190 con 0x7f9b780eacf0 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b78054930 con 0x7f9b78005e80 2024-03-29T04:09:34.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b780eacf0 0x7f9b78044ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7e7fc640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b78006800 0x7f9b78036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 -- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b78005e80 msgr2=0x7f9b780df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b78005e80 0x7f9b780df080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 -- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b78006800 msgr2=0x7f9b78036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b78006800 0x7f9b78036d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f9b7f7fe640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b780372c0 con 0x7f9b780eacf0 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa269800640 1 -- 172.21.15.181:0/4123980067 >> 172.21.15.181:0/4123980067 conn(0x7fa2640d0ee0 msgr2=0x7fa2640d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd6c0057d0 con 0x7efd6c009e20 2024-03-29T04:09:34.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 -- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 msgr2=0x7fb5640490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 0x7fb5640490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b7f7fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b780eacf0 0x7f9b78044ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b800511f0 tx=0x7f9b80074420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 -- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 msgr2=0x7fb564036510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 0x7fb564036510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7fa96b59e640 1 Processor -- start 2024-03-29T04:09:34.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fa96b59e640 1 -- start start 2024-03-29T04:09:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb564034d00 con 0x7fb564033980 2024-03-29T04:09:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fe932f4d640 1 Processor -- start 2024-03-29T04:09:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fe932f4d640 1 -- start start 2024-03-29T04:09:34.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fa269800640 1 -- 172.21.15.181:0/4123980067 shutdown_connections 2024-03-29T04:09:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8006dc60 con 0x7f9b780eacf0 2024-03-29T04:09:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8008f070 con 0x7f9b780eacf0 2024-03-29T04:09:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8006de60 con 0x7f9b780eacf0 2024-03-29T04:09:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b780438c0 con 0x7f9b780eacf0 2024-03-29T04:09:34.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fa269800640 1 -- 172.21.15.181:0/4123980067 wait complete. 2024-03-29T04:09:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b78043e80 con 0x7f9b780eacf0 2024-03-29T04:09:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd680319f0 con 0x7efd6c009e20 2024-03-29T04:09:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7efd5ffff640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efd5806c600 0x7efd5806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.857+0000 7f629a4c0640 1 -- 172.21.15.181:0/1241790416 wait complete. 2024-03-29T04:09:34.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f629a4c0640 1 Processor -- start 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56ad9d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb564033060 0x7fb5640490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb56a59c640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb564033980 0x7fb5640344d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c410 tx=0x7fb55c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0074e0 con 0x7fb564033980 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c0047e0 con 0x7fb564033980 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c004bd0 con 0x7fb564033980 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.858+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3dc001b070 con 0x7f3dc40b3e00 2024-03-29T04:09:34.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f3db9ffb640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3da806c600 0x7f3da806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3dc008af90 con 0x7f3dc40b3e00 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3da806c600 0x7f3da806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fa96b59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa964055400 0x7fa9640e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fa96b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9640557e0 con 0x7fa9640e7640 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fa96b59e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa964057f70 con 0x7fa964053c00 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fa96b59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa964056c80 con 0x7fa964055400 2024-03-29T04:09:34.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efd680ad910 con 0x7efd6c009e20 2024-03-29T04:09:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fb569d9b640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb564038170 0x7fb564036510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.859+0000 7f629a4c0640 1 -- start start 2024-03-29T04:09:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7f629a4c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62940c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7f629a4c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62940c5e70 0x7f629414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7efd5ffff640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7efd58075d70 0x7efd580781b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7f629a4c0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294071e30 0x7f62940722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa964055400 0x7fa9640e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa964055400 0x7fa9640e7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34466/0 (socket says 172.21.15.181:34466) 2024-03-29T04:09:34.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 -- 172.21.15.181:0/576685860 learned_addr learned my addr 172.21.15.181:0/576685860 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c200a70 0x7fe92c15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 0x7fe92c0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe92c2046e0 0x7fe92c204b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c0caed0 con 0x7fe92c072c20 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c15e830 con 0x7fe92c2046e0 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.860+0000 7fe932f4d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe92c0cc9d0 con 0x7fe92c200a70 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93174a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 0x7fe92c0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93174a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 0x7fe92c0730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34470/0 (socket says 172.21.15.181:34470) 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93174a640 1 -- 172.21.15.181:0/2731983451 learned_addr learned my addr 172.21.15.181:0/2731983451 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93274c640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe92c2046e0 0x7fe92c204b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c200a70 0x7fe92c15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b60001030 con 0x7f9b780eacf0 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7efd73fff640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7efd58075d70 0x7efd580781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd68004230 con 0x7efd58075d70 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efd680add00 con 0x7efd6c009e20 2024-03-29T04:09:34.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f6298cbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62940c5e70 0x7f629414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62940c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62940c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51190/0 (socket says 172.21.15.181:51190) 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f6298cbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62940c5e70 0x7f629414af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:53992/0 (socket says 172.21.15.181:53992) 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f629a4c0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f629415b440 con 0x7f6294071e30 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f629a4c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f629415b2c0 con 0x7f62940c5e70 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f629a4c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f629415b5c0 con 0x7f62800095d0 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 -- 172.21.15.181:0/4014019958 learned_addr learned my addr 172.21.15.181:0/4014019958 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f6299cbf640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294071e30 0x7f62940722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa269800640 1 Processor -- start 2024-03-29T04:09:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 -- 172.21.15.181:0/4014019958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62940c5e70 msgr2=0x7f629414af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62940c5e70 0x7f629414af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 -- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294071e30 msgr2=0x7f62940722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f62994be640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294071e30 0x7f62940722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93174a640 1 -- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c200a70 msgr2=0x7fe92c15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fe93174a640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe92c200a70 0x7fe92c15fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fe93174a640 1 -- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe92c2046e0 msgr2=0x7fe92c204b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fe93174a640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe92c2046e0 0x7fe92c204b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fe93174a640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c15f110 con 0x7fe92c072c20 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fe93274c640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe92c2046e0 0x7fe92c204b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f62994be640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62940b2a90 con 0x7f62800095d0 2024-03-29T04:09:34.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b80074c20 con 0x7f9b780eacf0 2024-03-29T04:09:34.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7efd73fff640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7efd58075d70 0x7efd580781b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7efd72ffd640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efd5806c600 0x7efd5806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f9b677fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6c06c600 0x7f9b6c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f62994be640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62940c3b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f629000d520 tx=0x7f629000d920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 -- 172.21.15.181:0/576685860 >> v1:172.21.15.187:6789/0 conn(0x7fa964053c00 legacy=0x7fa964006230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:34.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 -- 172.21.15.181:0/576685860 >> v1:172.21.15.181:6789/0 conn(0x7fa9640e7640 legacy=0x7fa9640534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fa96ad9d640 1 -- 172.21.15.181:0/576685860 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa964056ed0 con 0x7fa964055400 2024-03-29T04:09:34.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290005480 con 0x7f62800095d0 2024-03-29T04:09:34.899 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6290005480 con 0x7f62800095d0 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b80106640 con 0x7f9b780eacf0 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f9b677fe640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f9b6c075d80 0x7f9b6c0781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fe93174a640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 0x7fe92c0730a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe92400e5d0 tx=0x7fe92400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== osd.2 v2:172.21.15.181:6816/55833287 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd68004230 con 0x7efd58075d70 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f9b7effd640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6c06c600 0x7f9b6c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b80099eb0 con 0x7f9b6c075d80 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b8010d050 con 0x7f9b780eacf0 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fa269800640 1 -- start start 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 0x7fa264156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa26406c260 0x7fa26415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62940c66a0 con 0x7f62800095d0 2024-03-29T04:09:34.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62940b3030 con 0x7f62800095d0 2024-03-29T04:09:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa264162970 0x7fa26415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2640b52c0 con 0x7fa26406bac0 2024-03-29T04:09:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2640b5140 con 0x7fa264162970 2024-03-29T04:09:34.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa269800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2640b5440 con 0x7fa26406c260 2024-03-29T04:09:34.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62900035f0 con 0x7f62800095d0 2024-03-29T04:09:34.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa263fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 0x7fa264156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa263fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 0x7fa264156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34490/0 (socket says 172.21.15.181:34490) 2024-03-29T04:09:34.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa263fff640 1 -- 172.21.15.181:0/2542976802 learned_addr learned my addr 172.21.15.181:0/2542976802 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f9b7effd640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6c06c600 0x7f9b6c06eac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007570 tx=0x7f9b700010c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f3db9ffb640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7f3da8075d00 0x7f3da8078140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dc001be50 con 0x7f3da8075d00 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dc0054e10 con 0x7f3dc40b3e00 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7f3dc98cf640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7f3da8075d00 0x7f3da8078140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7f3dc88cd640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3da806c600 0x7f3da806eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc036000 tx=0x7f3dbc031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fa268fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa264162970 0x7fa26415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7efd72ffd640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efd5806c600 0x7efd5806eac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd7406a8c0 tx=0x7efd7406b4b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb564034fe0 con 0x7fb564033980 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.861+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb564034a10 con 0x7fb564033980 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.862+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb564035500 con 0x7fb564033980 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.863+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb55c011070 con 0x7fb564033980 2024-03-29T04:09:34.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7f9b7e7fc640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f9b6c075d80 0x7f9b6c0781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 -- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa26406c260 msgr2=0x7fa26415b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa26406c260 0x7fa26415b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 -- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa264162970 msgr2=0x7fa26415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa264162970 0x7fa26415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2641f8430 con 0x7fa26406bac0 2024-03-29T04:09:34.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924019070 con 0x7fe92c072c20 2024-03-29T04:09:34.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fb55affd640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb54c06c580 0x7fb54c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa96ad9d640 1 --2- 172.21.15.181:0/576685860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964055400 0x7fa9640e7a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02d940 tx=0x7fa96c0a7130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57025beedbbb3fef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7f3dc98cf640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7f3da8075d00 0x7f3da8078140 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.907 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa263fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 0x7fa264156360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa25400b3e0 tx=0x7fa25400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6294075890 con 0x7f62800095d0 2024-03-29T04:09:34.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb55c015e10 con 0x7fb564033980 2024-03-29T04:09:34.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c0c4110 con 0x7fe92c072c20 2024-03-29T04:09:34.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c0c46d0 con 0x7fe92c072c20 2024-03-29T04:09:34.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fb55affd640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7fb54c076950 0x7fb54c078df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb54c0794c0 con 0x7fb54c076950 2024-03-29T04:09:34.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa96959a640 1 -- 172.21.15.181:0/576685860 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0b4070 con 0x7fa964055400 2024-03-29T04:09:34.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa96959a640 1 -- 172.21.15.181:0/576685860 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c0af440 con 0x7fa964055400 2024-03-29T04:09:34.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa96959a640 1 -- 172.21.15.181:0/576685860 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0af8b0 con 0x7fa964055400 2024-03-29T04:09:34.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fb569d9b640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb54c06c580 0x7fb54c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.865+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb55c055bc0 con 0x7fb564033980 2024-03-29T04:09:34.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254015020 con 0x7fa26406bac0 2024-03-29T04:09:34.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa254011850 con 0x7fa26406bac0 2024-03-29T04:09:34.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254011c40 con 0x7fa26406bac0 2024-03-29T04:09:34.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2641f86f0 con 0x7fa26406bac0 2024-03-29T04:09:34.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2641f8cb0 con 0x7fa26406bac0 2024-03-29T04:09:34.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.865+0000 7fb56ad9d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7fb54c076950 0x7fb54c078df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.865+0000 7f9b7e7fc640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f9b6c075d80 0x7f9b6c0781c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92400ee90 con 0x7fe92c072c20 2024-03-29T04:09:34.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.864+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924019070 con 0x7fe92c072c20 2024-03-29T04:09:34.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.865+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== osd.5 v2:172.21.15.187:6820/461978944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dc001be50 con 0x7f3da8075d00 2024-03-29T04:09:34.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.865+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa264156960 con 0x7fa26406bac0 2024-03-29T04:09:34.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fb56ad9d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7fb54c076950 0x7fb54c078df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 -- 172.21.15.181:0/576685860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964055400 msgr2=0x7fa9640e7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 --2- 172.21.15.181:0/576685860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964055400 0x7fa9640e7a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02d940 tx=0x7fa96c0a7130 comp rx=0 tx=0).stop 2024-03-29T04:09:34.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fb569d9b640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb54c06c580 0x7fb54c06ea40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb564054250 tx=0x7fb56c09a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 -- 172.21.15.181:0/576685860 shutdown_connections 2024-03-29T04:09:34.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 --2- 172.21.15.181:0/576685860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964055400 0x7fa9640e7a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.914 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 -- 172.21.15.181:0/576685860 >> 172.21.15.181:0/576685860 conn(0x7fa96403aa20 msgr2=0x7fa96403ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.914 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe914001030 con 0x7fe92c072c20 2024-03-29T04:09:34.914 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== osd.1 v2:172.21.15.181:6808/3877871096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b80099eb0 con 0x7f9b6c075d80 2024-03-29T04:09:34.914 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 -- 172.21.15.181:0/576685860 shutdown_connections 2024-03-29T04:09:34.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6290003790 con 0x7f62800095d0 2024-03-29T04:09:34.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7fa96b59e640 1 -- 172.21.15.181:0/576685860 wait complete. 2024-03-29T04:09:34.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.866+0000 7f6289ffb640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f627806c760 0x7f627806ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7f6298cbd640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f627806c760 0x7f627806ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe92401c070 con 0x7fe92c072c20 2024-03-29T04:09:34.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f629008b500 con 0x7f62800095d0 2024-03-29T04:09:34.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7f6298cbd640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f627806c760 0x7f627806ec20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62940c9fe0 tx=0x7f6284009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== osd.4 v2:172.21.15.187:6800/1327517117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb54c0794c0 con 0x7fb54c076950 2024-03-29T04:09:34.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe9227fc640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe91006c730 0x7fe91006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fa96b59e640 1 Processor -- start 2024-03-29T04:09:34.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fa96b59e640 1 -- start start 2024-03-29T04:09:34.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7f6289ffb640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f6278075de0 0x7f6278078220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.867+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62780788f0 con 0x7f6278075de0 2024-03-29T04:09:34.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6290092050 con 0x7f62800095d0 2024-03-29T04:09:34.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7f6299cbf640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f6278075de0 0x7f6278078220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa254018020 con 0x7fa26406bac0 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fa260ff9640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa25006c600 0x7fa25006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa25408bf50 con 0x7fa26406bac0 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fa260ff9640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fa250075d70 0x7fa2500781b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa254018ec0 con 0x7fa250075d70 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa25408c340 con 0x7fa26406bac0 2024-03-29T04:09:34.919 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7f6299cbf640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f6278075de0 0x7f6278078220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 0x7fa96404d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa964055400 0x7fa96417d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640e7640 0x7fa96404ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa964057490 con 0x7fa964053c00 2024-03-29T04:09:34.920 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 0x7fa96404d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 0x7fa96404d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34500/0 (socket says 172.21.15.181:34500) 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96a59c640 1 -- 172.21.15.181:0/585389470 learned_addr learned my addr 172.21.15.181:0/585389470 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe92408c310 con 0x7fe92c072c20 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe91006c730 0x7fe91006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe9227fc640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7fe910075e30 0x7fe910078270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe924015e50 con 0x7fe910075e30 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe92408c7a0 con 0x7fe92c072c20 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.868+0000 7fe93274c640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7fe910075e30 0x7fe910078270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fe931f4b640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe91006c730 0x7fe91006ebf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe91c036000 tx=0x7fe91c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.869+0000 7fe93274c640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7fe910075e30 0x7fe910078270 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96ad9d640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640e7640 0x7fa96404ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa964058ce0 con 0x7fa9640e7640 2024-03-29T04:09:34.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== osd.0 v2:172.21.15.181:6800/3549016579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe924015e50 con 0x7fe910075e30 2024-03-29T04:09:34.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa2637fe640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa25006c600 0x7fa25006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.870+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa964057770 con 0x7fa964055400 2024-03-29T04:09:34.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa268fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fa250075d70 0x7fa2500781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa969d9b640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa964055400 0x7fa96417d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa2637fe640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa25006c600 0x7fa25006eac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa258036000 tx=0x7fa258008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== osd.3 v2:172.21.15.181:6824/3471664788 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62780788f0 con 0x7f6278075de0 2024-03-29T04:09:34.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa268fff640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fa250075d70 0x7fa2500781b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96a59c640 1 -- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa964055400 msgr2=0x7fa96417d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96a59c640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa964055400 0x7fa96417d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96a59c640 1 -- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640e7640 msgr2=0x7fa96404ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96a59c640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640e7640 0x7fa96404ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96a59c640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96404d820 con 0x7fa964053c00 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa96ad9d640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9640e7640 0x7fa96404ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.871+0000 7fa969d9b640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa964055400 0x7fa96417d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.872+0000 7fa96a59c640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 0x7fa96404d2e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c390 tx=0x7fa95c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.872+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c017070 con 0x7fa964053c00 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.872+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa964051db0 con 0x7fa964053c00 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.872+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c0052d0 con 0x7fa964053c00 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.873+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c003830 con 0x7fa964053c00 2024-03-29T04:09:34.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.873+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa964052370 con 0x7fa964053c00 2024-03-29T04:09:34.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.874+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== osd.6 v2:172.21.15.187:6808/3602434128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa254018ec0 con 0x7fa250075d70 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.874+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa95c01a070 con 0x7fa964053c00 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.875+0000 7fa95affd640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa94c06c600 0x7fa94c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.875+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa95c08ad10 con 0x7fa964053c00 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.876+0000 7fa969d9b640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa94c06c600 0x7fa94c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.876+0000 7fa969d9b640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa94c06c600 0x7fa94c06eac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9640e88b0 tx=0x7fa960009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.881+0000 7fa96b59e640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7fa96404cc70 0x7fa96404f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.881+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa96404f9c0 con 0x7fa96404cc70 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.881+0000 7fa96ad9d640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7fa96404cc70 0x7fa96404f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.895+0000 7fa96ad9d640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7fa96404cc70 0x7fa96404f480 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:34.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.896+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 --> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd6c005d90 con 0x7efd58075d70 2024-03-29T04:09:34.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.896+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== osd.7 v2:172.21.15.187:6815/235935836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa96404f9c0 con 0x7fa96404cc70 2024-03-29T04:09:34.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.897+0000 7efd5ffff640 1 -- 172.21.15.181:0/1586521859 <== osd.2 v2:172.21.15.181:6816/55833287 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efd6c005d90 con 0x7efd58075d70 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.897+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7efd58075d70 msgr2=0x7efd580781b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.897+0000 7efd79f24640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7efd58075d70 0x7efd580781b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efd5806c600 msgr2=0x7efd5806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efd5806c600 0x7efd5806eac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd7406a8c0 tx=0x7efd7406b4b0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 msgr2=0x7efd6c187190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 --2- 172.21.15.181:0/1586521859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd6c009e20 0x7efd6c187190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd68004000 tx=0x7efd68004730 comp rx=0 tx=0).stop 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd73fff640 1 -- 172.21.15.181:0/1586521859 reap_dead start 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 shutdown_connections 2024-03-29T04:09:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 >> 172.21.15.181:0/1586521859 conn(0x7efd6c0537a0 msgr2=0x7efd6c0e1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 shutdown_connections 2024-03-29T04:09:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.898+0000 7efd79f24640 1 -- 172.21.15.181:0/1586521859 wait complete. 2024-03-29T04:09:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.899+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 --> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b60002dd0 con 0x7f9b6c075d80 2024-03-29T04:09:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.900+0000 7f9b677fe640 1 -- 172.21.15.181:0/3527032522 <== osd.1 v2:172.21.15.181:6808/3877871096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b60002dd0 con 0x7f9b6c075d80 2024-03-29T04:09:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.899+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 --> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3dc40b28e0 con 0x7f3da8075d00 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f9b6c075d80 msgr2=0x7f9b6c0781c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f9b6c075d80 0x7f9b6c0781c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6c06c600 msgr2=0x7f9b6c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6c06c600 0x7f9b6c06eac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007570 tx=0x7f9b700010c0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b780eacf0 msgr2=0x7f9b78044ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f9b85740640 1 --2- 172.21.15.181:0/3527032522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b780eacf0 0x7f9b78044ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b800511f0 tx=0x7f9b80074420 comp rx=0 tx=0).stop 2024-03-29T04:09:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.901+0000 7f3db9ffb640 1 -- 172.21.15.181:0/3590196233 <== osd.5 v2:172.21.15.187:6820/461978944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3dc40b28e0 con 0x7f3da8075d00 2024-03-29T04:09:34.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.902+0000 7f9b7f7fe640 1 -- 172.21.15.181:0/3527032522 reap_dead start 2024-03-29T04:09:34.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.902+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 shutdown_connections 2024-03-29T04:09:34.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.902+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 >> 172.21.15.181:0/3527032522 conn(0x7f9b78009fa0 msgr2=0x7f9b78049390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.902+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 shutdown_connections 2024-03-29T04:09:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.902+0000 7f9b85740640 1 -- 172.21.15.181:0/3527032522 wait complete. 2024-03-29T04:09:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.903+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7f3da8075d00 msgr2=0x7f3da8078140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.903+0000 7f3dcb332640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7f3da8075d00 0x7f3da8078140 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.903+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 --> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa2640af140 con 0x7fa250075d70 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3da806c600 msgr2=0x7f3da806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7f3dcb332640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3da806c600 0x7f3da806eac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc036000 tx=0x7f3dbc031000 comp rx=0 tx=0).stop 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 msgr2=0x7f3dc414ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7f3dcb332640 1 --2- 172.21.15.181:0/3590196233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3dc40b3e00 0x7f3dc414ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000c750 tx=0x7f3dc000cc10 comp rx=0 tx=0).stop 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7f3dc98cf640 1 -- 172.21.15.181:0/3590196233 reap_dead start 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 --> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe914002dd0 con 0x7fe910075e30 2024-03-29T04:09:34.935 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 shutdown_connections 2024-03-29T04:09:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 >> 172.21.15.181:0/3590196233 conn(0x7f3dc40b2b00 msgr2=0x7f3dc4161790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.904+0000 7fa260ff9640 1 -- 172.21.15.181:0/2542976802 <== osd.6 v2:172.21.15.187:6808/3602434128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa2640af140 con 0x7fa250075d70 2024-03-29T04:09:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7fe9227fc640 1 -- 172.21.15.181:0/2731983451 <== osd.0 v2:172.21.15.181:6800/3549016579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe914002dd0 con 0x7fe910075e30 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 shutdown_connections 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7f3dcb332640 1 -- 172.21.15.181:0/3590196233 wait complete. 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fa250075d70 msgr2=0x7fa2500781b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.905+0000 7fa269800640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fa250075d70 0x7fa2500781b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa25006c600 msgr2=0x7fa25006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa25006c600 0x7fa25006eac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa258036000 tx=0x7fa258008030 comp rx=0 tx=0).stop 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 msgr2=0x7fa264156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 --2- 172.21.15.181:0/2542976802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26406bac0 0x7fa264156360 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa25400b3e0 tx=0x7fa25400b8a0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.937 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa268fff640 1 -- 172.21.15.181:0/2542976802 reap_dead start 2024-03-29T04:09:34.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7fe910075e30 msgr2=0x7fe910078270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7fe910075e30 0x7fe910078270 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 shutdown_connections 2024-03-29T04:09:34.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 >> 172.21.15.181:0/2542976802 conn(0x7fa2640d0ee0 msgr2=0x7fa2640b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe91006c730 msgr2=0x7fe91006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe91006c730 0x7fe91006ebf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe91c036000 tx=0x7fe91c030000 comp rx=0 tx=0).stop 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 msgr2=0x7fe92c0730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 --2- 172.21.15.181:0/2731983451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe92c072c20 0x7fe92c0730a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe92400e5d0 tx=0x7fe92400e6c0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe93274c640 1 -- 172.21.15.181:0/2731983451 reap_dead start 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 shutdown_connections 2024-03-29T04:09:34.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 >> 172.21.15.181:0/2731983451 conn(0x7fe92c0ccc80 msgr2=0x7fe92c0ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 shutdown_connections 2024-03-29T04:09:34.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 shutdown_connections 2024-03-29T04:09:34.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 --> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62940b58d0 con 0x7f6278075de0 2024-03-29T04:09:34.940 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 --> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb56404f760 con 0x7fb54c076950 2024-03-29T04:09:34.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.906+0000 7fa269800640 1 -- 172.21.15.181:0/2542976802 wait complete. 2024-03-29T04:09:34.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7fb55affd640 1 -- 172.21.15.181:0/3868488495 <== osd.4 v2:172.21.15.187:6800/1327517117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb56404f760 con 0x7fb54c076950 2024-03-29T04:09:34.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7f6289ffb640 1 -- 172.21.15.181:0/4014019958 <== osd.3 v2:172.21.15.181:6824/3471664788 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62940b58d0 con 0x7f6278075de0 2024-03-29T04:09:34.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.908+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f6278075de0 msgr2=0x7f6278078220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.908+0000 7f629a4c0640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f6278075de0 0x7f6278078220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.942 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.908+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7fb54c076950 msgr2=0x7fb54c078df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.945 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.908+0000 7fb570d6d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7fb54c076950 0x7fb54c078df0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb54c06c580 msgr2=0x7fb54c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7fb570d6d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb54c06c580 0x7fb54c06ea40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb564054250 tx=0x7fb56c09a000 comp rx=0 tx=0).stop 2024-03-29T04:09:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb564033980 msgr2=0x7fb5640344d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7fb570d6d640 1 --2- 172.21.15.181:0/3868488495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb564033980 0x7fb5640344d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c410 tx=0x7fb55c00c8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.946 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7fb56ad9d640 1 -- 172.21.15.181:0/3868488495 reap_dead start 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f627806c760 msgr2=0x7f627806ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f627806c760 0x7f627806ec20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62940c9fe0 tx=0x7f6284009190 comp rx=0 tx=0).stop 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 msgr2=0x7f62940c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 --2- 172.21.15.181:0/4014019958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62800095d0 0x7f62940c3b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f629000d520 tx=0x7f629000d920 comp rx=0 tx=0).stop 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f6299cbf640 1 -- 172.21.15.181:0/4014019958 reap_dead start 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 shutdown_connections 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 >> 172.21.15.181:0/4014019958 conn(0x7f62940b1800 msgr2=0x7f62940afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.907+0000 7fe932f4d640 1 -- 172.21.15.181:0/2731983451 wait complete. 2024-03-29T04:09:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 shutdown_connections 2024-03-29T04:09:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.909+0000 7f629a4c0640 1 -- 172.21.15.181:0/4014019958 wait complete. 2024-03-29T04:09:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.910+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 shutdown_connections 2024-03-29T04:09:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.910+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 >> 172.21.15.181:0/3868488495 conn(0x7fb564009fa0 msgr2=0x7fb56404f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.910+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 shutdown_connections 2024-03-29T04:09:34.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.910+0000 7fb570d6d640 1 -- 172.21.15.181:0/3868488495 wait complete. 2024-03-29T04:09:34.949 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:09:34.950 INFO:teuthology.orchestra.run.smithi181.stdout:55834574857 2024-03-29T04:09:34.951 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:09:34.951 INFO:teuthology.orchestra.run.smithi181.stdout:55834574859 2024-03-29T04:09:34.952 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:09:34.953 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:09:34.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.931+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 --> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9640382a0 con 0x7fa96404cc70 2024-03-29T04:09:34.954 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.954 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:09:34.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.931+0000 7fa95affd640 1 -- 172.21.15.181:0/585389470 <== osd.7 v2:172.21.15.187:6815/235935836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9640382a0 con 0x7fa96404cc70 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7fa96404cc70 msgr2=0x7fa96404f480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7fa96404cc70 0x7fa96404f480 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa94c06c600 msgr2=0x7fa94c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa94c06c600 0x7fa94c06eac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9640e88b0 tx=0x7fa960009040 comp rx=0 tx=0).stop 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 msgr2=0x7fa96404d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 --2- 172.21.15.181:0/585389470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa964053c00 0x7fa96404d2e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c390 tx=0x7fa95c00c850 comp rx=0 tx=0).stop 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96ad9d640 1 -- 172.21.15.181:0/585389470 reap_dead start 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 shutdown_connections 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 >> 172.21.15.181:0/585389470 conn(0x7fa96403aa20 msgr2=0x7fa96403a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 shutdown_connections 2024-03-29T04:09:34.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:34.932+0000 7fa96b59e640 1 -- 172.21.15.181:0/585389470 wait complete. 2024-03-29T04:09:34.957 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:09:34.958 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.958 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:09:34.959 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:34.959 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:09:35.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.182+0000 7fe7181db640 1 Processor -- start 2024-03-29T04:09:35.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.181+0000 7f190edf0640 1 Processor -- start 2024-03-29T04:09:35.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.182+0000 7f6da6255640 1 Processor -- start 2024-03-29T04:09:35.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.182+0000 7f6da6255640 1 -- start start 2024-03-29T04:09:35.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.182+0000 7f190edf0640 1 -- start start 2024-03-29T04:09:35.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f7c84396640 1 Processor -- start 2024-03-29T04:09:35.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f190edf0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f190806f150 0x7f1908165580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f190edf0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190806c3a0 con 0x7f19080ccc90 2024-03-29T04:09:35.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f190edf0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190806f650 con 0x7f190815aae0 2024-03-29T04:09:35.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f190edf0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f190815aec0 con 0x7f190806f150 2024-03-29T04:09:35.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f94ad401640 1 Processor -- start 2024-03-29T04:09:35.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f190cb8c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f190815aae0 0x7f190806b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35072/0 (socket says 172.21.15.181:35072) 2024-03-29T04:09:35.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f190cb8c640 1 -- 172.21.15.181:0/1514239337 learned_addr learned my addr 172.21.15.181:0/1514239337 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:35.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da6255640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d980df210 0x7f6d98038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f190d38d640 1 --2- 172.21.15.181:0/1514239337 >> v2:172.21.15.187:3300/0 conn(0x7f190806f150 0x7f1908165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da6255640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 0x7f6d9804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da5253640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d980df210 0x7f6d98038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7fe7181db640 1 -- start start 2024-03-29T04:09:35.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f7c84396640 1 -- start start 2024-03-29T04:09:35.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da6255640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d980544b0 con 0x7f6d98033ee0 2024-03-29T04:09:35.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da6255640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f6d980541d0 con 0x7f6d98053c60 2024-03-29T04:09:35.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f190d38d640 1 -- 172.21.15.181:0/1514239337 >> v1:172.21.15.187:6789/0 conn(0x7f190815aae0 legacy=0x7f190806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f190d38d640 1 -- 172.21.15.181:0/1514239337 >> v1:172.21.15.181:6789/0 conn(0x7f19080ccc90 legacy=0x7f190806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f190d38d640 1 -- 172.21.15.181:0/1514239337 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19080cb1a0 con 0x7f190806f150 2024-03-29T04:09:35.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da6255640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6d98005f20 con 0x7f6d980df210 2024-03-29T04:09:35.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da5253640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d980df210 0x7f6d98038070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54050/0 (socket says 172.21.15.181:54050) 2024-03-29T04:09:35.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7f6da5253640 1 -- 172.21.15.181:0/443048343 learned_addr learned my addr 172.21.15.181:0/443048343 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.183+0000 7f94ad401640 1 -- start start 2024-03-29T04:09:35.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f6da4a52640 1 --2- 172.21.15.181:0/443048343 >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 0x7f6d9804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94ad401640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94a004fa80 0x7f94a004fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c84396640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 0x7f7c74051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c84396640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c74048510 con 0x7f7c74008160 2024-03-29T04:09:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c84396640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c740487b0 con 0x7f7c740548a0 2024-03-29T04:09:35.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c84396640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7c74048a70 con 0x7f7c7404fc00 2024-03-29T04:09:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f6da5253640 1 -- 172.21.15.181:0/443048343 >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 msgr2=0x7f6d9804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94ad401640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 0x7f94a00389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94ad401640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a00503a0 con 0x7f94a00df410 2024-03-29T04:09:35.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94ad401640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f94a0006890 con 0x7f94a004fa80 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94ad401640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f94a0006ae0 con 0x7f94a0054260 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94a67fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94a00df410 0x7f94a00382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57110/0 (socket says 172.21.15.181:57110) 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94a67fc640 1 -- 172.21.15.181:0/4032465177 learned_addr learned my addr 172.21.15.181:0/4032465177 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94a6ffd640 1 --2- 172.21.15.181:0/4032465177 >> v2:172.21.15.181:3300/0 conn(0x7f94a004fa80 0x7f94a004fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f94a77fe640 1 --2- 172.21.15.181:0/4032465177 >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 0x7f94a00389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7fbcced8d640 1 Processor -- start 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c81931640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7c74008160 0x7f7c7404ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57126/0 (socket says 172.21.15.181:57126) 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c81931640 1 -- 172.21.15.181:0/2879211120 learned_addr learned my addr 172.21.15.181:0/2879211120 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7f7c82132640 1 --2- 172.21.15.181:0/2879211120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 0x7f7c74051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f7c82132640 1 -- 172.21.15.181:0/2879211120 >> v1:172.21.15.187:6789/0 conn(0x7f7c740548a0 legacy=0x7f7c740f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f7c82132640 1 -- 172.21.15.181:0/2879211120 >> v1:172.21.15.181:6789/0 conn(0x7f7c74008160 legacy=0x7f7c7404ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f7c82132640 1 -- 172.21.15.181:0/2879211120 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c74053090 con 0x7f7c7404fc00 2024-03-29T04:09:35.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f94a6ffd640 1 -- 172.21.15.181:0/4032465177 >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 msgr2=0x7f94a00389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f94a6ffd640 1 --2- 172.21.15.181:0/4032465177 >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 0x7f94a00389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f94a6ffd640 1 -- 172.21.15.181:0/4032465177 >> v1:172.21.15.181:6789/0 conn(0x7f94a00df410 legacy=0x7f94a00382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f94a6ffd640 1 -- 172.21.15.181:0/4032465177 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a0006dc0 con 0x7f94a004fa80 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7fbcced8d640 1 -- start start 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f94a77fe640 1 --2- 172.21.15.181:0/4032465177 >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 0x7f94a00389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f6da5253640 1 --2- 172.21.15.181:0/443048343 >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 0x7f6d9804fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f6da5253640 1 -- 172.21.15.181:0/443048343 >> v1:172.21.15.181:6789/0 conn(0x7f6d98033ee0 legacy=0x7f6d98006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f6da5253640 1 -- 172.21.15.181:0/443048343 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d98006200 con 0x7f6d980df210 2024-03-29T04:09:35.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f6da4a52640 1 --2- 172.21.15.181:0/443048343 >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 0x7f6d9804fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:09:35.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f190d38d640 1 --2- 172.21.15.181:0/1514239337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190806f150 0x7f1908165580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c00 tx=0x7f1900030090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f1dbb043563561 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f62c4e5b640 1 Processor -- start 2024-03-29T04:09:35.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f19077fe640 1 -- 172.21.15.181:0/1514239337 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190003d070 con 0x7f190806f150 2024-03-29T04:09:35.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f19077fe640 1 -- 172.21.15.181:0/1514239337 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1900030e40 con 0x7f190806f150 2024-03-29T04:09:35.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f62c4e5b640 1 -- start start 2024-03-29T04:09:35.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f19077fe640 1 -- 172.21.15.181:0/1514239337 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900038660 con 0x7f190806f150 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eac60 0x7fe708052670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe70803b0a0 0x7fe7080483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe7080eb040 0x7fe7080488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe708052bb0 con 0x7fe7080eb040 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe70803a090 con 0x7fe70803b0a0 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe7181db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe70803ac10 con 0x7fe7080eac60 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe715776640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe70803b0a0 0x7fe7080483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.184+0000 7fe715776640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe70803b0a0 0x7fe7080483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54058/0 (socket says 172.21.15.181:54058) 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 -- 172.21.15.181:0/2485759202 learned_addr learned my addr 172.21.15.181:0/2485759202 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 -- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eac60 msgr2=0x7fe708052670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 --2- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eac60 0x7fe708052670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 -- 172.21.15.181:0/2485759202 >> v2:172.21.15.181:3300/0 conn(0x7fe7080eb040 msgr2=0x7fe7080488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 --2- 172.21.15.181:0/2485759202 >> v2:172.21.15.181:3300/0 conn(0x7fe7080eb040 0x7fe7080488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.185+0000 7fe715776640 1 -- 172.21.15.181:0/2485759202 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70803a2e0 con 0x7fe70803b0a0 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fe715776640 1 --2- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080483b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dca0 tx=0x7fe70c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c577423b1b60b118 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f4586111640 1 Processor -- start 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.186+0000 7f4586111640 1 -- start start 2024-03-29T04:09:35.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f4586111640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45800ccbc0 0x7f458015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fe714f75640 1 -- 172.21.15.181:0/2485759202 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0315a0 con 0x7fe70803b0a0 2024-03-29T04:09:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fe714f75640 1 -- 172.21.15.181:0/2485759202 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c037070 con 0x7fe70803b0a0 2024-03-29T04:09:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fe714f75640 1 -- 172.21.15.181:0/2485759202 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0315a0 con 0x7fe70803b0a0 2024-03-29T04:09:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcced8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcced8d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc8166820 0x7fbcc80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f457ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45800ccbc0 0x7f458015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f457ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45800ccbc0 0x7f458015eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54090/0 (socket says 172.21.15.181:54090) 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f457ffff640 1 -- 172.21.15.181:0/1341036059 learned_addr learned my addr 172.21.15.181:0/1341036059 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f4586111640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45800cad90 con 0x7f458015b6e0 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458015ac00 con 0x7f458015e700 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f458015a9d0 con 0x7f45800ccbc0 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcccb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcced8d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc80cf4c0 con 0x7fbcc80c2e80 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcced8d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc80cd480 con 0x7fbcc8166820 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcced8d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc8164100 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcccb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51240/0 (socket says 172.21.15.181:51240) 2024-03-29T04:09:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7fbcccb29640 1 -- 172.21.15.181:0/3664266252 learned_addr learned my addr 172.21.15.181:0/3664266252 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f7c82132640 1 --2- 172.21.15.181:0/2879211120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 0x7f7c74051760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7c780097d0 tx=0x7f7c7802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c62afe61490c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f94a6ffd640 1 --2- 172.21.15.181:0/4032465177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a004fa80 0x7f94a004fe60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94a000dc50 tx=0x7f949802dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24073bcc9db196de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.187+0000 7f7c81130640 1 -- 172.21.15.181:0/2879211120 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78034070 con 0x7f7c7404fc00 2024-03-29T04:09:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fbcc7fff640 1 --2- 172.21.15.181:0/3664266252 >> v2:172.21.15.181:3300/0 conn(0x7fbcc8166820 0x7fbcc80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f6da5253640 1 --2- 172.21.15.181:0/443048343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98038070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b090 tx=0x7f6d9c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96416e50eac508 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f94a5ffb640 1 -- 172.21.15.181:0/4032465177 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949802f8a0 con 0x7f94a004fa80 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f7c81130640 1 -- 172.21.15.181:0/2879211120 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78030800 con 0x7f7c7404fc00 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62c4e5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 0x7f62b8058580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62c4e5b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62b8058ac0 0x7f62b8053990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62c4e5b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b80574e0 con 0x7f62b80e3390 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62c4e5b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f62b8054780 con 0x7f62b8058ac0 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62c4e5b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62b80e3770 con 0x7f62b80382a0 2024-03-29T04:09:35.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 -- 172.21.15.181:0/1514239337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190806f150 msgr2=0x7f1908165580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62be59c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f62b80e3390 0x7f62b8056d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57138/0 (socket says 172.21.15.181:57138) 2024-03-29T04:09:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f6d93fff640 1 -- 172.21.15.181:0/443048343 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c035020 con 0x7f6d980df210 2024-03-29T04:09:35.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62be59c640 1 -- 172.21.15.181:0/3868464928 learned_addr learned my addr 172.21.15.181:0/3868464928 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 --2- 172.21.15.181:0/1514239337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190806f150 0x7f1908165580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c00 tx=0x7f1900030090 comp rx=0 tx=0).stop 2024-03-29T04:09:35.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f6d93fff640 1 -- 172.21.15.181:0/443048343 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d9c031870 con 0x7f6d980df210 2024-03-29T04:09:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/3868464928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 0x7f62b8058580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f6d93fff640 1 -- 172.21.15.181:0/443048343 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c035020 con 0x7f6d980df210 2024-03-29T04:09:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62bf59e640 1 --2- 172.21.15.181:0/3868464928 >> v2:172.21.15.181:3300/0 conn(0x7f62b8058ac0 0x7f62b8053990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 -- 172.21.15.181:0/1514239337 shutdown_connections 2024-03-29T04:09:35.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 --2- 172.21.15.181:0/1514239337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190806f150 0x7f1908165580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 -- 172.21.15.181:0/1514239337 >> 172.21.15.181:0/1514239337 conn(0x7f19080cd150 msgr2=0x7f19080cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fe7181db640 1 -- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 msgr2=0x7fe7080483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fe7181db640 1 --2- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080483b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dca0 tx=0x7fe70c02b840 comp rx=0 tx=0).stop 2024-03-29T04:09:35.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f190edf0640 1 -- 172.21.15.181:0/1514239337 shutdown_connections 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62bdd9b640 1 -- 172.21.15.181:0/3868464928 >> v2:172.21.15.181:3300/0 conn(0x7f62b8058ac0 msgr2=0x7f62b8053990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/3868464928 >> v2:172.21.15.181:3300/0 conn(0x7f62b8058ac0 0x7f62b8053990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f457ffff640 1 -- 172.21.15.181:0/1341036059 >> v1:172.21.15.187:6789/0 conn(0x7f458015e700 legacy=0x7f45800ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f457ffff640 1 -- 172.21.15.181:0/1341036059 >> v1:172.21.15.181:6789/0 conn(0x7f458015b6e0 legacy=0x7f45800cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f190edf0640 1 -- 172.21.15.181:0/1514239337 wait complete. 2024-03-29T04:09:35.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 -- 172.21.15.181:0/2485759202 shutdown_connections 2024-03-29T04:09:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 --2- 172.21.15.181:0/2485759202 >> v2:172.21.15.181:3300/0 conn(0x7fe7080eb040 0x7fe7080488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 --2- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080483b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 --2- 172.21.15.181:0/2485759202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eac60 0x7fe708052670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 -- 172.21.15.181:0/2485759202 >> 172.21.15.181:0/2485759202 conn(0x7fe70801e6b0 msgr2=0x7fe7080553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f62bdd9b640 1 -- 172.21.15.181:0/3868464928 >> v1:172.21.15.181:6789/0 conn(0x7f62b80e3390 legacy=0x7f62b8056d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f62bdd9b640 1 -- 172.21.15.181:0/3868464928 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80577c0 con 0x7f62b80382a0 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fbcccb29640 1 -- 172.21.15.181:0/3664266252 >> v2:172.21.15.181:3300/0 conn(0x7fbcc8166820 msgr2=0x7fbcc80ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fbcccb29640 1 --2- 172.21.15.181:0/3664266252 >> v2:172.21.15.181:3300/0 conn(0x7fbcc8166820 0x7fbcc80ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fbcccb29640 1 -- 172.21.15.181:0/3664266252 >> v1:172.21.15.187:6789/0 conn(0x7fbcc80c2e80 legacy=0x7fbcc80d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7fbcccb29640 1 -- 172.21.15.181:0/3664266252 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc8166de0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f94a5ffb640 1 -- 172.21.15.181:0/4032465177 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949802fa40 con 0x7f94a004fa80 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.188+0000 7f94a5ffb640 1 -- 172.21.15.181:0/4032465177 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949802fe10 con 0x7f94a004fa80 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 -- 172.21.15.181:0/4032465177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a004fa80 msgr2=0x7f94a004fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 --2- 172.21.15.181:0/4032465177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a004fa80 0x7f94a004fe60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94a000dc50 tx=0x7f949802dae0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 -- 172.21.15.181:0/4032465177 shutdown_connections 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 --2- 172.21.15.181:0/4032465177 >> v2:172.21.15.187:3300/0 conn(0x7f94a0054260 0x7f94a00389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 --2- 172.21.15.181:0/4032465177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a004fa80 0x7f94a004fe60 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94a000dc50 tx=0x7f949802dae0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f94ad401640 1 -- 172.21.15.181:0/4032465177 >> 172.21.15.181:0/4032465177 conn(0x7f94a0009fa0 msgr2=0x7f94a000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fbcccb29640 1 --2- 172.21.15.181:0/3664266252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37f980f2db466a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fbcc77fe640 1 -- 172.21.15.181:0/3664266252 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc002cb00 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f7c84396640 1 -- 172.21.15.181:0/2879211120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 msgr2=0x7f7c74051760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f7c84396640 1 --2- 172.21.15.181:0/2879211120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 0x7f7c74051760 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7c780097d0 tx=0x7f7c7802b940 comp rx=0 tx=0).stop 2024-03-29T04:09:35.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fbcc77fe640 1 -- 172.21.15.181:0/3664266252 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc002cca0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7f457ffff640 1 -- 172.21.15.181:0/1341036059 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458015aee0 con 0x7f45800ccbc0 2024-03-29T04:09:35.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 -- 172.21.15.181:0/2485759202 shutdown_connections 2024-03-29T04:09:35.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fe7181db640 1 -- 172.21.15.181:0/2485759202 wait complete. 2024-03-29T04:09:35.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/3868464928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 0x7f62b8058580 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097a0 tx=0x7f62b002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de04f612b769974 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 -- 172.21.15.181:0/443048343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 msgr2=0x7f6d98038070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f7c84396640 1 -- 172.21.15.181:0/2879211120 shutdown_connections 2024-03-29T04:09:35.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f7c84396640 1 --2- 172.21.15.181:0/2879211120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c7404fc00 0x7f7c74051760 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f7c84396640 1 -- 172.21.15.181:0/2879211120 >> 172.21.15.181:0/2879211120 conn(0x7f7c74055340 msgr2=0x7f7c74055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f94ad401640 1 -- 172.21.15.181:0/4032465177 shutdown_connections 2024-03-29T04:09:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 --2- 172.21.15.181:0/443048343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98038070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b090 tx=0x7f6d9c02b910 comp rx=0 tx=0).stop 2024-03-29T04:09:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f7c84396640 1 -- 172.21.15.181:0/2879211120 shutdown_connections 2024-03-29T04:09:35.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 -- 172.21.15.181:0/443048343 shutdown_connections 2024-03-29T04:09:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 --2- 172.21.15.181:0/443048343 >> v2:172.21.15.181:3300/0 conn(0x7f6d98053c60 0x7f6d9804fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 --2- 172.21.15.181:0/443048343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98038070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f94ad401640 1 -- 172.21.15.181:0/4032465177 wait complete. 2024-03-29T04:09:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 -- 172.21.15.181:0/443048343 >> 172.21.15.181:0/443048343 conn(0x7f6d98009fa0 msgr2=0x7f6d9800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.189+0000 7fbcc77fe640 1 -- 172.21.15.181:0/3664266252 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0034610 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f457ffff640 1 --2- 172.21.15.181:0/1341036059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45800ccbc0 0x7f458015eae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f457800c310 tx=0x7f457802d420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1028dd919566232 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 -- 172.21.15.181:0/443048343 shutdown_connections 2024-03-29T04:09:35.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f6da6255640 1 -- 172.21.15.181:0/443048343 wait complete. 2024-03-29T04:09:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f190edf0640 1 Processor -- start 2024-03-29T04:09:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 -- 172.21.15.181:0/3664266252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 msgr2=0x7fbcc80ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 --2- 172.21.15.181:0/3664266252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc002bf60 comp rx=0 tx=0).stop 2024-03-29T04:09:35.203 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f190edf0640 1 -- start start 2024-03-29T04:09:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f62bd59a640 1 -- 172.21.15.181:0/3868464928 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0034070 con 0x7f62b80382a0 2024-03-29T04:09:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f62bd59a640 1 -- 172.21.15.181:0/3868464928 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0030610 con 0x7f62b80382a0 2024-03-29T04:09:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f62bd59a640 1 -- 172.21.15.181:0/3868464928 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00309e0 con 0x7f62b80382a0 2024-03-29T04:09:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fe7181db640 1 Processor -- start 2024-03-29T04:09:35.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f457effd640 1 -- 172.21.15.181:0/1341036059 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578039070 con 0x7f45800ccbc0 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f457effd640 1 -- 172.21.15.181:0/1341036059 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457800a4e0 con 0x7f45800ccbc0 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f457effd640 1 -- 172.21.15.181:0/1341036059 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457800a8b0 con 0x7f45800ccbc0 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 -- 172.21.15.181:0/3664266252 shutdown_connections 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 --2- 172.21.15.181:0/3664266252 >> v2:172.21.15.181:3300/0 conn(0x7fbcc8166820 0x7fbcc80ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 --2- 172.21.15.181:0/3664266252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80cf0e0 0x7fbcc80ce130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7fbcced8d640 1 -- 172.21.15.181:0/3664266252 >> 172.21.15.181:0/3664266252 conn(0x7fbcc806d610 msgr2=0x7fbcc806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.191+0000 7fe7181db640 1 -- start start 2024-03-29T04:09:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.191+0000 7fbcced8d640 1 -- 172.21.15.181:0/3664266252 shutdown_connections 2024-03-29T04:09:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.191+0000 7fbcced8d640 1 -- 172.21.15.181:0/3664266252 wait complete. 2024-03-29T04:09:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.191+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45800ccbc0 msgr2=0x7f458015eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.191+0000 7f4586111640 1 --2- 172.21.15.181:0/1341036059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45800ccbc0 0x7f458015eae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f457800c310 tx=0x7f457802d420 comp rx=0 tx=0).stop 2024-03-29T04:09:35.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f94ad401640 1 Processor -- start 2024-03-29T04:09:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 shutdown_connections 2024-03-29T04:09:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f4586111640 1 --2- 172.21.15.181:0/1341036059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45800ccbc0 0x7f458015eae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 >> 172.21.15.181:0/1341036059 conn(0x7f45800b19e0 msgr2=0x7f45800b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.190+0000 7f7c84396640 1 -- 172.21.15.181:0/2879211120 wait complete. 2024-03-29T04:09:35.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f7c84396640 1 Processor -- start 2024-03-29T04:09:35.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f94ad401640 1 -- start start 2024-03-29T04:09:35.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f7c84396640 1 -- start start 2024-03-29T04:09:35.208 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7fe7181db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7fe7181db640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 0x7fe7080c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 0x7f19081fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190cb8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190cb8c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51268/0 (socket says 172.21.15.181:51268) 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190cb8c640 1 -- 172.21.15.181:0/770244795 learned_addr learned my addr 172.21.15.181:0/770244795 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190815aae0 0x7f19080c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f190806f930 con 0x7f19080ccc90 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f6da6255640 1 Processor -- start 2024-03-29T04:09:35.210 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19080cb760 con 0x7f190815aae0 2024-03-29T04:09:35.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7fbcced8d640 1 Processor -- start 2024-03-29T04:09:35.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f190edf0640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19080ced40 con 0x7f190806f150 2024-03-29T04:09:35.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f6da6255640 1 -- start start 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f1907fff640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 0x7f19081fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f62c4e5b640 1 -- 172.21.15.181:0/3868464928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 msgr2=0x7f62b8058580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/3868464928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 0x7f62b8058580 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097a0 tx=0x7f62b002b960 comp rx=0 tx=0).stop 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190d38d640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190815aae0 0x7f19080c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 -- 172.21.15.181:0/3868464928 shutdown_connections 2024-03-29T04:09:35.212 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/3868464928 >> v2:172.21.15.181:3300/0 conn(0x7f62b8058ac0 0x7f62b8053990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/3868464928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b80382a0 0x7f62b8058580 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 -- 172.21.15.181:0/3868464928 >> 172.21.15.181:0/3868464928 conn(0x7f62b8059130 msgr2=0x7f62b8059510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190cb8c640 1 -- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190815aae0 msgr2=0x7f19080c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190cb8c640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190815aae0 0x7f19080c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190cb8c640 1 -- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 msgr2=0x7f19081fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.213 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c84396640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c84396640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 0x7f7c7404b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c84396640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190cb8c640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 0x7f19081fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f190cb8c640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19080c9290 con 0x7f190806f150 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f1907fff640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 0x7f19081fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 -- 172.21.15.181:0/3868464928 shutdown_connections 2024-03-29T04:09:35.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f62c4e5b640 1 -- 172.21.15.181:0/3868464928 wait complete. 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe7181db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eb040 0x7fe7080e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe7181db640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe708055fc0 con 0x7fe7080eac60 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 shutdown_connections 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7f4586111640 1 -- 172.21.15.181:0/1341036059 wait complete. 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe715f77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.215 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe715f77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54102/0 (socket says 172.21.15.181:54102) 2024-03-29T04:09:35.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe715f77640 1 -- 172.21.15.181:0/317836550 learned_addr learned my addr 172.21.15.181:0/317836550 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f4586111640 1 Processor -- start 2024-03-29T04:09:35.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe70803a8a0 con 0x7fe70803b0a0 2024-03-29T04:09:35.216 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c82132640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe708056140 con 0x7fe7080eb040 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7fe715776640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 0x7fe7080c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94a0054260 0x7f94a00df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c82132640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51270/0 (socket says 172.21.15.181:51270) 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c81931640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 0x7f7c7404b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.217 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.193+0000 7f7c81931640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 0x7f7c7404b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54114/0 (socket says 172.21.15.181:54114) 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da6255640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d98033ee0 0x7f6d980c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94a00df410 0x7f94a00485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0052b30 con 0x7f94a00d1630 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94a0007380 con 0x7f94a00df410 2024-03-29T04:09:35.218 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94ad401640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94a0052cb0 con 0x7f94a0054260 2024-03-29T04:09:35.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f190cb8c640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c750 tx=0x7f18fc00cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da6255640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c82933640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.219 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c82933640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34562/0 (socket says 172.21.15.181:34562) 2024-03-29T04:09:35.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fe715f77640 1 -- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eb040 msgr2=0x7fe7080e8480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:35.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fe715f77640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eb040 0x7fe7080e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c84396640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740f2880 con 0x7f7c740548a0 2024-03-29T04:09:35.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fe715f77640 1 -- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 msgr2=0x7fe7080c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c84396640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740506f0 con 0x7f7c7404fc00 2024-03-29T04:09:35.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c84396640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740f2a00 con 0x7f7c74008160 2024-03-29T04:09:35.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34576/0 (socket says 172.21.15.181:34576) 2024-03-29T04:09:35.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f94a77fe640 1 -- 172.21.15.181:0/87017603 learned_addr learned my addr 172.21.15.181:0/87017603 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f7c82132640 1 -- 172.21.15.181:0/1287259641 learned_addr learned my addr 172.21.15.181:0/1287259641 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da5253640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d98033ee0 0x7f6d980c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da4a52640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc018070 con 0x7f190806f150 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.192+0000 7fbcced8d640 1 -- start start 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 0x7fbcc80c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 0x7fbcc80c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcccb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 0x7fbcc80c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcccb29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 0x7fbcc80c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51282/0 (socket says 172.21.15.181:51282) 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcccb29640 1 -- 172.21.15.181:0/1420795719 learned_addr learned my addr 172.21.15.181:0/1420795719 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcc7fff640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 0x7fbcc80c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8166820 0x7fbcc8150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80cfd10 con 0x7fbcc8166820 2024-03-29T04:09:35.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81673a0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc8167680 con 0x7fbcc80c2e80 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da4a52640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51300/0 (socket says 172.21.15.181:51300) 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da4a52640 1 -- 172.21.15.181:0/2539322075 learned_addr learned my addr 172.21.15.181:0/2539322075 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f6da6255640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98049440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da6255640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98037a80 con 0x7f6d98033ee0 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da6255640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98037900 con 0x7f6d980df210 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da6255640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d980df5f0 con 0x7f6d98053c60 2024-03-29T04:09:35.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 -- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 msgr2=0x7f7c7404b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 0x7f7c7404b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 -- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 msgr2=0x7f7c74034d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 -- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94a0054260 msgr2=0x7f94a00df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94a0054260 0x7f94a00df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 -- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94a00df410 msgr2=0x7f94a00485e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94a00df410 0x7f94a00485e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 -- 172.21.15.181:0/87017603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a00e2440 con 0x7f94a00d1630 2024-03-29T04:09:35.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f190edf0640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19081fcf30 con 0x7f190806f150 2024-03-29T04:09:35.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbccd32a640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8166820 0x7fbcc8150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da5a54640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98049440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 -- 172.21.15.181:0/1287259641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c74035c80 con 0x7f7c74008160 2024-03-29T04:09:35.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.194+0000 7f4586111640 1 -- start start 2024-03-29T04:09:35.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f4586111640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82933640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:35.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f190edf0640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19080c9af0 con 0x7f190806f150 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f94a77fe640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e560 tx=0x7f949c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 -- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 msgr2=0x7fbcc80c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 0x7fbcc80c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 -- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8166820 msgr2=0x7fbcc8150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8166820 0x7fbcc8150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc80c7700 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 -- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 msgr2=0x7f6d98049440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98049440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 -- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d98033ee0 msgr2=0x7f6d980c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d98033ee0 0x7f6d980c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9804d650 con 0x7f6d98053c60 2024-03-29T04:09:35.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f7c82132640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c780098f0 tx=0x7f7c780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f62c4e5b640 1 Processor -- start 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f62c4e5b640 1 -- start start 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fbcc7fff640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 0x7fbcc80c87a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcb801e560 tx=0x7fbcb801ea20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fe715f77640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 0x7fe7080c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fe715f77640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7080e9f60 con 0x7fe70803b0a0 2024-03-29T04:09:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7fe715776640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 0x7fe7080c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:35.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f6da4a52640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c820 tx=0x7f6d9400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.195+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00f040 con 0x7f94a00d1630 2024-03-29T04:09:35.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c004230 con 0x7f94a00d1630 2024-03-29T04:09:35.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb801f040 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78034040 con 0x7f7c74008160 2024-03-29T04:09:35.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7802f560 con 0x7f7c74008160 2024-03-29T04:09:35.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a004cde0 con 0x7f94a00d1630 2024-03-29T04:09:35.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb802a070 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc80c79e0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a004d3a0 con 0x7f94a00d1630 2024-03-29T04:09:35.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc80c7fa0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c003610 con 0x7f94a00d1630 2024-03-29T04:09:35.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7fe715f77640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b450 tx=0x7fe70000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94015070 con 0x7f6d98053c60 2024-03-29T04:09:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94007a30 con 0x7f6d98053c60 2024-03-29T04:09:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94007de0 con 0x7f6d98053c60 2024-03-29T04:09:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f62c4e5b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 0x7f62b8189760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.196+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7802f950 con 0x7f7c74008160 2024-03-29T04:09:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f6da6255640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d98048680 con 0x7f6d98053c60 2024-03-29T04:09:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c74035f60 con 0x7f7c74008160 2024-03-29T04:09:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7404aea0 con 0x7f7c74008160 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700015020 con 0x7fe70803b0a0 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7000118b0 con 0x7fe70803b0a0 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700011ca0 con 0x7fe70803b0a0 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f6da6255640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d98048c40 con 0x7f6d98053c60 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7080e96c0 con 0x7fe70803b0a0 2024-03-29T04:09:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7080e9c80 con 0x7fe70803b0a0 2024-03-29T04:09:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62c4e5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62c4e5b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 0x7f62b8050200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc0072f0 con 0x7f190806f150 2024-03-29T04:09:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0047e0 con 0x7f190806f150 2024-03-29T04:09:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62c4e5b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8058060 con 0x7f62b80e3390 2024-03-29T04:09:35.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62c4e5b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8057d80 con 0x7f62b80382a0 2024-03-29T04:09:35.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f18fc01b070 con 0x7f190806f150 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62c4e5b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62b8053ed0 con 0x7f62b8058ac0 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62bdd9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62bdd9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51306/0 (socket says 172.21.15.181:51306) 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62bdd9b640 1 -- 172.21.15.181:0/253670222 learned_addr learned my addr 172.21.15.181:0/253670222 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f62be59c640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 0x7f62b8189760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62bf59e640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 0x7f62b8050200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62be59c640 1 -- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 msgr2=0x7f62b80522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62be59c640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62be59c640 1 -- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 msgr2=0x7f62b8050200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f4586111640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f458015b6e0 0x7f45800cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c005290 con 0x7f6d98053c60 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62be59c640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 0x7f62b8050200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f62be59c640 1 -- 172.21.15.181:0/253670222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b8189f90 con 0x7f62b80382a0 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:35.240 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f62bf59e640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 0x7f62b8050200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9488005290 con 0x7f94a00d1630 2024-03-29T04:09:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f19057fa640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 0x7f18ec06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18fc08afd0 con 0x7f190806f150 2024-03-29T04:09:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f1907fff640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 0x7f18ec06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e8005290 con 0x7f190806f150 2024-03-29T04:09:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f62be59c640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 0x7f62b8189760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62b4000f60 tx=0x7f62b400c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c74051510 con 0x7f7c74008160 2024-03-29T04:09:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7fbcaeffd640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc8075900 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8025860 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbcb8025a00 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f4586111640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 0x7f45800c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.197+0000 7f4586111640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45800caa50 con 0x7f45800ccbc0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f457f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f457f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34604/0 (socket says 172.21.15.181:34604) 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f457f7fe640 1 -- 172.21.15.181:0/3075248586 learned_addr learned my addr 172.21.15.181:0/3075248586 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f45800cb070 con 0x7f458015e700 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.198+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f458006f2a0 con 0x7f458015b6e0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f457ffff640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 0x7f45800c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f4576ffd640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f458015b6e0 0x7f45800cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4017070 con 0x7f62b80382a0 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f1907fff640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 0x7f18ec06eaf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1908156a00 tx=0x7f18f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6d94018070 con 0x7f6d98053c60 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b818a270 con 0x7f62b80382a0 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7fbcc57fa640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 0x7fbcb406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbcb809c9d0 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7fbcccb29640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 0x7fbcb406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f6d91ffb640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 0x7f6d8406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6d9408b980 con 0x7f6d98053c60 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.199+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f949c0037b0 con 0x7f94a00d1630 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f948f7fe640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 0x7f949406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f949c08b620 con 0x7f94a00d1630 2024-03-29T04:09:35.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b8189ca0 con 0x7f62b80382a0 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b4005ce0 con 0x7f62b80382a0 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b40072d0 con 0x7f62b80382a0 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457ffff640 1 -- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f458015b6e0 msgr2=0x7f45800cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457ffff640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f458015b6e0 0x7f45800cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457ffff640 1 -- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 msgr2=0x7f45800c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457ffff640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457ffff640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45800c4ea0 con 0x7f458015e700 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.200+0000 7f457f7fe640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:35.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7f457ffff640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 0x7f45800c5f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4578033000 tx=0x7f4578004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7fbcccb29640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 0x7fbcb406eac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009850 tx=0x7fbcc0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7f94a6ffd640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 0x7f949406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe700018020 con 0x7fe70803b0a0 2024-03-29T04:09:35.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7c78037070 con 0x7f7c74008160 2024-03-29T04:09:35.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7f6da5253640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 0x7f6d8406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.201+0000 7f6da5253640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 0x7f6d8406eac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c036000 tx=0x7f6d9c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7fe7067fc640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 0x7fe6f406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f94a6ffd640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 0x7f949406eac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9498038000 tx=0x7f9498032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe70008baa0 con 0x7fe70803b0a0 2024-03-29T04:09:35.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7fe715776640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 0x7fe6f406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7080057d0 con 0x7fe70803b0a0 2024-03-29T04:09:35.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f7c72ffd640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 0x7f7c6006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a4005290 con 0x7f62b80382a0 2024-03-29T04:09:35.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f7c81931640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 0x7f7c6006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7c780ade30 con 0x7f7c74008160 2024-03-29T04:09:35.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7fe715776640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 0x7fe6f406ea40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe70c036000 tx=0x7fe70c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.202+0000 7f7c81931640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 0x7f7c6006ea40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c740dc740 tx=0x7f7c6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.204+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578043020 con 0x7f458015e700 2024-03-29T04:09:35.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.204+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbcb806ad20 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.204+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45800c5140 con 0x7f458015e700 2024-03-29T04:09:35.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.204+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45800c59e0 con 0x7f458015e700 2024-03-29T04:09:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.205+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f62b401a070 con 0x7f62b80382a0 2024-03-29T04:09:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.205+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45800c5310 con 0x7f458015e700 2024-03-29T04:09:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.205+0000 7f62aeffd640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 0x7f62a006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.206+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f949c017020 con 0x7f94a00d1630 2024-03-29T04:09:35.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.206+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 0x7f62a006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.206+0000 7f62bdd9b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 0x7f62a006ea40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009560 tx=0x7f62b003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.208+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f62b401de30 con 0x7f62b80382a0 2024-03-29T04:09:35.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.208+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457800a420 con 0x7f458015e700 2024-03-29T04:09:35.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.209+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578039040 con 0x7f458015e700 2024-03-29T04:09:35.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.209+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe70001f020 con 0x7fe70803b0a0 2024-03-29T04:09:35.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.209+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6d940595f0 con 0x7f6d98053c60 2024-03-29T04:09:35.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.213+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f62b4058d90 con 0x7f62b80382a0 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.214+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4578046020 con 0x7f458015e700 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.214+0000 7f457cff9640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 0x7f456006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.215+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45780ae960 con 0x7f458015e700 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.216+0000 7f457f7fe640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 0x7f456006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.216+0000 7f457f7fe640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 0x7f456006e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4570001b90 tx=0x7f4570009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.218+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f457804d030 con 0x7f458015e700 2024-03-29T04:09:35.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.219+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f18fc058c40 con 0x7f190806f150 2024-03-29T04:09:35.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.224+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7c74051510 con 0x7f7c74008160 2024-03-29T04:09:35.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.605+0000 7fbcaeffd640 1 -- 172.21.15.181:0/1420795719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbcc8071700 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.606+0000 7fbcc57fa640 1 -- 172.21.15.181:0/1420795719 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcb806a620 con 0x7fbcc80cf0e0 2024-03-29T04:09:35.585 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:35.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.609+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 msgr2=0x7fbcb406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.609+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 0x7fbcb406eac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009850 tx=0x7fbcc0030000 comp rx=0 tx=0).stop 2024-03-29T04:09:35.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.609+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 msgr2=0x7fbcc80c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.609+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 0x7fbcc80c87a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcb801e560 tx=0x7fbcb801ea20 comp rx=0 tx=0).stop 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 shutdown_connections 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc8166820 0x7fbcc8150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcc80cf0e0 0x7fbcc80c87a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbcb406c600 0x7fbcb406eac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 --2- 172.21.15.181:0/1420795719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbcc80c2e80 0x7fbcc80c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 >> 172.21.15.181:0/1420795719 conn(0x7fbcc806d610 msgr2=0x7fbcc8162570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 shutdown_connections 2024-03-29T04:09:35.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.610+0000 7fbcced8d640 1 -- 172.21.15.181:0/1420795719 wait complete. 2024-03-29T04:09:35.606 INFO:teuthology.orchestra.run.smithi181.stdout:55834574856 2024-03-29T04:09:35.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.624+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9488005580 con 0x7f94a00d1630 2024-03-29T04:09:35.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.625+0000 7f948f7fe640 1 -- 172.21.15.181:0/87017603 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f949c0598e0 con 0x7f94a00d1630 2024-03-29T04:09:35.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-03-29T04:09:35.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.627+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6d7c005580 con 0x7f6d98053c60 2024-03-29T04:09:35.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.627+0000 7f6d91ffb640 1 -- 172.21.15.181:0/2539322075 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d9405d8f0 con 0x7f6d98053c60 2024-03-29T04:09:35.608 INFO:teuthology.orchestra.run.smithi181.stdout:55834574856 2024-03-29T04:09:35.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.627+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe7080d82d0 con 0x7fe70803b0a0 2024-03-29T04:09:35.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.629+0000 7fe7067fc640 1 -- 172.21.15.181:0/317836550 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe70001f020 con 0x7fe70803b0a0 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stdout:55834574855 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 msgr2=0x7f6d8406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 0x7f6d8406eac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c036000 tx=0x7f6d9c03c040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 msgr2=0x7f6d9804e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c820 tx=0x7f6d9400cc20 comp rx=0 tx=0).stop 2024-03-29T04:09:35.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 shutdown_connections 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 msgr2=0x7f949406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d980df210 0x7f6d98049440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f6d8406c600 0x7f6d8406eac0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c036000 tx=0x7f6d9c03c040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d98053c60 0x7f6d9804e270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 --2- 172.21.15.181:0/2539322075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d98033ee0 0x7f6d980c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 >> 172.21.15.181:0/2539322075 conn(0x7f6d98009fa0 msgr2=0x7f6d980dfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 0x7f949406eac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9498038000 tx=0x7f9498032000 comp rx=0 tx=0).stop 2024-03-29T04:09:35.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 msgr2=0x7f94a017c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e560 tx=0x7f949c00ea20 comp rx=0 tx=0).stop 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 shutdown_connections 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a00d1630 0x7f94a017c860 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94a00df410 0x7f94a00485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f949406c600 0x7f949406eac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 --2- 172.21.15.181:0/87017603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94a0054260 0x7f94a00df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 >> 172.21.15.181:0/87017603 conn(0x7f94a0009fa0 msgr2=0x7f94a000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 shutdown_connections 2024-03-29T04:09:35.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f6d837fe640 1 -- 172.21.15.181:0/2539322075 wait complete. 2024-03-29T04:09:35.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 shutdown_connections 2024-03-29T04:09:35.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.630+0000 7f94ad401640 1 -- 172.21.15.181:0/87017603 wait complete. 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 msgr2=0x7fe6f406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 0x7fe6f406ea40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe70c036000 tx=0x7fe70c03d040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 msgr2=0x7fe7080d4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b450 tx=0x7fe70000b910 comp rx=0 tx=0).stop 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 shutdown_connections 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe6f406c580 0x7fe6f406ea40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7080eb040 0x7fe7080e8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7080eac60 0x7fe7080c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 --2- 172.21.15.181:0/317836550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe70803b0a0 0x7fe7080d4c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f62a4005850 con 0x7f62b80382a0 2024-03-29T04:09:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 >> 172.21.15.181:0/317836550 conn(0x7fe70801e6b0 msgr2=0x7fe708053d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 shutdown_connections 2024-03-29T04:09:35.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7fe7181db640 1 -- 172.21.15.181:0/317836550 wait complete. 2024-03-29T04:09:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.632+0000 7f62aeffd640 1 -- 172.21.15.181:0/253670222 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62b405d090 con 0x7f62b80382a0 2024-03-29T04:09:35.616 INFO:teuthology.orchestra.run.smithi181.stdout:55834574857 2024-03-29T04:09:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 msgr2=0x7f62a006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 0x7f62a006ea40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009560 tx=0x7f62b003b040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 msgr2=0x7f62b8189760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 0x7f62b8189760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62b4000f60 tx=0x7f62b400c760 comp rx=0 tx=0).stop 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 shutdown_connections 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62b80e3390 0x7f62b8050200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f62a006c580 0x7f62a006ea40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62b8058ac0 0x7f62b80522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 --2- 172.21.15.181:0/253670222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b80382a0 0x7f62b8189760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.636+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 >> 172.21.15.181:0/253670222 conn(0x7f62b8059130 msgr2=0x7f62b80073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.638+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 shutdown_connections 2024-03-29T04:09:35.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.638+0000 7f62c4e5b640 1 -- 172.21.15.181:0/253670222 wait complete. 2024-03-29T04:09:35.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.643+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7c7400c940 con 0x7f7c74008160 2024-03-29T04:09:35.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.643+0000 7f7c72ffd640 1 -- 172.21.15.181:0/1287259641 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c7400c940 con 0x7f7c74008160 2024-03-29T04:09:35.623 INFO:teuthology.orchestra.run.smithi181.stdout:55834574856 2024-03-29T04:09:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.646+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 msgr2=0x7f7c6006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 0x7f7c6006ea40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7c740dc740 tx=0x7f7c6c009040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-03-29T04:09:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 msgr2=0x7f7c74185e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c780098f0 tx=0x7f7c780046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 shutdown_connections 2024-03-29T04:09:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c740548a0 0x7f7c74034d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c7404fc00 0x7f7c7404b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7c6006c580 0x7f7c6006ea40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 --2- 172.21.15.181:0/1287259641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7c74008160 0x7f7c74185e10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 >> 172.21.15.181:0/1287259641 conn(0x7f7c74055340 msgr2=0x7f7c74047960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 shutdown_connections 2024-03-29T04:09:35.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.647+0000 7f7c84396640 1 -- 172.21.15.181:0/1287259641 wait complete. 2024-03-29T04:09:35.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-03-29T04:09:35.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2024-03-29T04:09:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.658+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f18e8005580 con 0x7f190806f150 2024-03-29T04:09:35.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.658+0000 7f19057fa640 1 -- 172.21.15.181:0/770244795 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18fc05cf40 con 0x7f190806f150 2024-03-29T04:09:35.638 INFO:teuthology.orchestra.run.smithi181.stdout:55834574856 2024-03-29T04:09:35.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 msgr2=0x7f18ec06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 0x7f18ec06eaf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1908156a00 tx=0x7f18f4009040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 msgr2=0x7f19080ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c750 tx=0x7f18fc00cc10 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 shutdown_connections 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f190815aae0 0x7f19080c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19080ccc90 0x7f19081fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f18ec06c630 0x7f18ec06eaf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 --2- 172.21.15.181:0/770244795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f190806f150 0x7f19080ca2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 >> 172.21.15.181:0/770244795 conn(0x7f19080cd150 msgr2=0x7f190806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 shutdown_connections 2024-03-29T04:09:35.641 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.660+0000 7f18f2ffd640 1 -- 172.21.15.181:0/770244795 wait complete. 2024-03-29T04:09:35.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.666+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4580075890 con 0x7f458015e700 2024-03-29T04:09:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.666+0000 7f457cff9640 1 -- 172.21.15.181:0/3075248586 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f457807c780 con 0x7f458015e700 2024-03-29T04:09:35.646 INFO:teuthology.orchestra.run.smithi181.stdout:55834574856 2024-03-29T04:09:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 msgr2=0x7f456006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 0x7f456006e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4570001b90 tx=0x7f4570009040 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 msgr2=0x7f45800c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 0x7f45800c5f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4578033000 tx=0x7f4578004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 shutdown_connections 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f458015e700 0x7f45800c5f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f456006c530 0x7f456006e9f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f458015b6e0 0x7f45800cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 --2- 172.21.15.181:0/3075248586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45800ccbc0 0x7f45800c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 >> 172.21.15.181:0/3075248586 conn(0x7f45800b19e0 msgr2=0x7f458015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 shutdown_connections 2024-03-29T04:09:35.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:35.669+0000 7f4586111640 1 -- 172.21.15.181:0/3075248586 wait complete. 2024-03-29T04:09:35.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-03-29T04:09:35.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.7 2024-03-29T04:09:36.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T04:09:36.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T04:09:36.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T04:09:36.633 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T04:09:36.639 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T04:09:36.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T04:09:36.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T04:09:36.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.767+0000 7f28e759e640 1 Processor -- start 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.768+0000 7f28e759e640 1 -- start start 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e759e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28e80cf540 0x7f28e8163440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e8166420 0x7f28e8163980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e759e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e81669e0 con 0x7f28e80ca890 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e759e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28e8166cc0 con 0x7f28e80cf540 2024-03-29T04:09:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e759e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28e8166fa0 con 0x7f28e8166420 2024-03-29T04:09:36.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.769+0000 7f28e5d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28e80cf540 0x7f28e8163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28e80cf540 0x7f28e8163440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54178/0 (socket says 172.21.15.181:54178) 2024-03-29T04:09:36.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 -- 172.21.15.181:0/4119124892 learned_addr learned my addr 172.21.15.181:0/4119124892 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e6d9d640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e8166420 0x7f28e8163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 -- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e8166420 msgr2=0x7f28e8163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e8166420 0x7f28e8163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 -- 172.21.15.181:0/4119124892 >> v1:172.21.15.181:6789/0 conn(0x7f28e80ca890 legacy=0x7f28e80711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.770+0000 7f28e5d9b640 1 -- 172.21.15.181:0/4119124892 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e8167280 con 0x7f28e80cf540 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.771+0000 7f28e5d9b640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e80cf540 0x7f28e8163440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009870 tx=0x7f28d802b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e40233810917f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.772+0000 7f28e559a640 1 -- 172.21.15.181:0/4119124892 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8034070 con 0x7f28e80cf540 2024-03-29T04:09:36.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.772+0000 7f28e559a640 1 -- 172.21.15.181:0/4119124892 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d8030880 con 0x7f28e80cf540 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 -- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e80cf540 msgr2=0x7f28e8163440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e80cf540 0x7f28e8163440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009870 tx=0x7f28d802b9c0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 -- 172.21.15.181:0/4119124892 shutdown_connections 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e8166420 0x7f28e8163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 --2- 172.21.15.181:0/4119124892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e80cf540 0x7f28e8163440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.773+0000 7f28e759e640 1 -- 172.21.15.181:0/4119124892 >> 172.21.15.181:0/4119124892 conn(0x7f28e80cc7b0 msgr2=0x7f28e80ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.774+0000 7f28e759e640 1 -- 172.21.15.181:0/4119124892 shutdown_connections 2024-03-29T04:09:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.774+0000 7f28e759e640 1 -- 172.21.15.181:0/4119124892 wait complete. 2024-03-29T04:09:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.776+0000 7f28e759e640 1 Processor -- start 2024-03-29T04:09:36.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.776+0000 7f28e759e640 1 -- start start 2024-03-29T04:09:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.777+0000 7f28e759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e8166420 0x7f28e814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.777+0000 7f28e759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e759e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e5d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e5d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34616/0 (socket says 172.21.15.181:34616) 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e6d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51340/0 (socket says 172.21.15.181:51340) 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e5d9b640 1 -- 172.21.15.181:0/2488547349 learned_addr learned my addr 172.21.15.181:0/2488547349 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80cac70 con 0x7f28e80b2950 2024-03-29T04:09:36.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80cc4d0 con 0x7f28e8166420 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.778+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28e80718e0 con 0x7f28e814c7c0 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e659c640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e8166420 0x7f28e814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 -- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e8166420 msgr2=0x7f28e814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e8166420 0x7f28e814d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 -- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 msgr2=0x7f28e814c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e80c67e0 con 0x7f28e814c7c0 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e6d9d640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28e0007d40 tx=0x7f28e000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.779+0000 7f28e5d9b640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.780+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0019070 con 0x7f28e814c7c0 2024-03-29T04:09:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.780+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e80c24c0 con 0x7f28e814c7c0 2024-03-29T04:09:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.780+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e8147400 con 0x7f28e814c7c0 2024-03-29T04:09:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.781+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e0003990 con 0x7f28e814c7c0 2024-03-29T04:09:36.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.781+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0003d60 con 0x7f28e814c7c0 2024-03-29T04:09:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.782+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc005290 con 0x7f28e814c7c0 2024-03-29T04:09:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.783+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f28e001c070 con 0x7f28e814c7c0 2024-03-29T04:09:36.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.784+0000 7f28d6ffd640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 0x7f28d006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.785+0000 7f28e659c640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 0x7f28d006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.786+0000 7f28e659c640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 0x7f28d006ec10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b2790 tx=0x7f28dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.786+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f28e008b9a0 con 0x7f28e814c7c0 2024-03-29T04:09:36.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.791+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f28e0059610 con 0x7f28e814c7c0 2024-03-29T04:09:36.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.837+0000 7f40797f3640 1 Processor -- start 2024-03-29T04:09:36.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.837+0000 7f40797f3640 1 -- start start 2024-03-29T04:09:36.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.838+0000 7f40797f3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f406c0df380 0x7f406c0ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.838+0000 7f40797f3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c006ae0 con 0x7f406c050190 2024-03-29T04:09:36.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.838+0000 7f40797f3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c006dc0 con 0x7f406c00aa90 2024-03-29T04:09:36.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.838+0000 7f40797f3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f406c006890 con 0x7f406c0df380 2024-03-29T04:09:36.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4073fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f406c00aa90 0x7f406c04fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35098/0 (socket says 172.21.15.181:35098) 2024-03-29T04:09:36.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4073fff640 1 -- 172.21.15.181:0/1928773312 learned_addr learned my addr 172.21.15.181:0/1928773312 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:36.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4078ff2640 1 --2- 172.21.15.181:0/1928773312 >> v2:172.21.15.187:3300/0 conn(0x7f406c0df380 0x7f406c0ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.838+0000 7f2232250640 1 Processor -- start 2024-03-29T04:09:36.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f2232250640 1 -- start start 2024-03-29T04:09:36.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4078ff2640 1 -- 172.21.15.181:0/1928773312 >> v1:172.21.15.187:6789/0 conn(0x7f406c00aa90 legacy=0x7f406c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4078ff2640 1 -- 172.21.15.181:0/1928773312 >> v1:172.21.15.181:6789/0 conn(0x7f406c050190 legacy=0x7f406c054260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.839+0000 7f4078ff2640 1 -- 172.21.15.181:0/1928773312 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0070a0 con 0x7f406c0df380 2024-03-29T04:09:36.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f2232250640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f2232250640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f222c0ce4d0 0x7f222c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f2232250640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222c0ca950 con 0x7f222c086e10 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f2232250640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f222c16c500 con 0x7f222c0ce4d0 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f2232250640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f222c0cac30 con 0x7f222c0b17b0 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f223124e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f223124e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51350/0 (socket says 172.21.15.181:51350) 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.840+0000 7f223124e640 1 -- 172.21.15.181:0/1600371000 learned_addr learned my addr 172.21.15.181:0/1600371000 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f2231a4f640 1 --2- 172.21.15.181:0/1600371000 >> v2:172.21.15.187:3300/0 conn(0x7f222c0ce4d0 0x7f222c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f2223fff640 1 -- 172.21.15.181:0/1600371000 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434779185 0 0) 0x7f222c0ca950 con 0x7f222c086e10 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f2223fff640 1 -- 172.21.15.181:0/1600371000 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2218003520 con 0x7f222c086e10 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f223124e640 1 -- 172.21.15.181:0/1600371000 >> v2:172.21.15.187:3300/0 conn(0x7f222c0ce4d0 msgr2=0x7f222c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f223124e640 1 --2- 172.21.15.181:0/1600371000 >> v2:172.21.15.187:3300/0 conn(0x7f222c0ce4d0 0x7f222c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f223124e640 1 -- 172.21.15.181:0/1600371000 >> v1:172.21.15.187:6789/0 conn(0x7f222c086e10 legacy=0x7f222c0b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f2223fff640 1 -- 172.21.15.181:0/1600371000 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2981763764 0 0) 0x7f2218003520 con 0x7f222c086e10 2024-03-29T04:09:36.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f223124e640 1 -- 172.21.15.181:0/1600371000 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c0cdcd0 con 0x7f222c0b17b0 2024-03-29T04:09:36.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f4078ff2640 1 --2- 172.21.15.181:0/1928773312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0ead40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40600097a0 tx=0x7f406002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd1ba6fe707f60f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.841+0000 7f949f2d9640 1 Processor -- start 2024-03-29T04:09:36.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f4072ffd640 1 -- 172.21.15.181:0/1928773312 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060037070 con 0x7f406c0df380 2024-03-29T04:09:36.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f4072ffd640 1 -- 172.21.15.181:0/1928773312 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4060030710 con 0x7f406c0df380 2024-03-29T04:09:36.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f4072ffd640 1 -- 172.21.15.181:0/1928773312 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060030ae0 con 0x7f406c0df380 2024-03-29T04:09:36.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f949f2d9640 1 -- start start 2024-03-29T04:09:36.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f223124e640 1 --2- 172.21.15.181:0/1600371000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f221c008870 tx=0x7f221c02de70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275861f319ac54e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f40797f3640 1 -- 172.21.15.181:0/1928773312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 msgr2=0x7f406c0ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949f2d9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94980b2740 0x7f949815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949f2d9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f949815f000 0x7f94980cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949f2d9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94980cb570 con 0x7f949815a9f0 2024-03-29T04:09:36.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f40797f3640 1 --2- 172.21.15.181:0/1928773312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0ead40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40600097a0 tx=0x7f406002b710 comp rx=0 tx=0).stop 2024-03-29T04:09:36.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949f2d9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f949815e520 con 0x7f949815f000 2024-03-29T04:09:36.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949f2d9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f949815e800 con 0x7f94980b2740 2024-03-29T04:09:36.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f2223fff640 1 -- 172.21.15.181:0/1600371000 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c034070 con 0x7f222c0b17b0 2024-03-29T04:09:36.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.842+0000 7f2223fff640 1 -- 172.21.15.181:0/1600371000 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c02e9a0 con 0x7f222c0b17b0 2024-03-29T04:09:36.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f40797f3640 1 -- 172.21.15.181:0/1928773312 shutdown_connections 2024-03-29T04:09:36.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f2232250640 1 -- 172.21.15.181:0/1600371000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 msgr2=0x7f222c16bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f2232250640 1 --2- 172.21.15.181:0/1600371000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f221c008870 tx=0x7f221c02de70 comp rx=0 tx=0).stop 2024-03-29T04:09:36.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f40797f3640 1 --2- 172.21.15.181:0/1928773312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0ead40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f40797f3640 1 -- 172.21.15.181:0/1928773312 >> 172.21.15.181:0/1928773312 conn(0x7f406c009fa0 msgr2=0x7f406c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949d075640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94980b2740 0x7f949815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949d075640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f94980b2740 0x7f949815a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54202/0 (socket says 172.21.15.181:54202) 2024-03-29T04:09:36.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949d075640 1 -- 172.21.15.181:0/3934850079 learned_addr learned my addr 172.21.15.181:0/3934850079 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.843+0000 7f949d876640 1 --2- 172.21.15.181:0/3934850079 >> v2:172.21.15.181:3300/0 conn(0x7f949815f000 0x7f94980cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f2232250640 1 -- 172.21.15.181:0/1600371000 shutdown_connections 2024-03-29T04:09:36.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f2232250640 1 --2- 172.21.15.181:0/1600371000 >> v2:172.21.15.187:3300/0 conn(0x7f222c0ce4d0 0x7f222c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f2232250640 1 --2- 172.21.15.181:0/1600371000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c16bfc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f2232250640 1 -- 172.21.15.181:0/1600371000 >> 172.21.15.181:0/1600371000 conn(0x7f222c0cc7b0 msgr2=0x7f222c0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f67997d6640 1 Processor -- start 2024-03-29T04:09:36.829 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f67997d6640 1 -- start start 2024-03-29T04:09:36.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f949d075640 1 -- 172.21.15.181:0/3934850079 >> v2:172.21.15.181:3300/0 conn(0x7f949815f000 msgr2=0x7f94980cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f949d075640 1 --2- 172.21.15.181:0/3934850079 >> v2:172.21.15.181:3300/0 conn(0x7f949815f000 0x7f94980cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f40797f3640 1 -- 172.21.15.181:0/1928773312 shutdown_connections 2024-03-29T04:09:36.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f40797f3640 1 -- 172.21.15.181:0/1928773312 wait complete. 2024-03-29T04:09:36.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f949d075640 1 -- 172.21.15.181:0/3934850079 >> v1:172.21.15.187:6789/0 conn(0x7f949815a9f0 legacy=0x7f94980caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.844+0000 7f949d075640 1 -- 172.21.15.181:0/3934850079 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949815eae0 con 0x7f94980b2740 2024-03-29T04:09:36.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f0ad4d31640 1 Processor -- start 2024-03-29T04:09:36.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f0ad4d31640 1 -- start start 2024-03-29T04:09:36.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f949d075640 1 --2- 172.21.15.181:0/3934850079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94980b2740 0x7f949815a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9498071080 tx=0x7f949002bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3a93552254bc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f948ffff640 1 -- 172.21.15.181:0/3934850079 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949002cb10 con 0x7f94980b2740 2024-03-29T04:09:36.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f948ffff640 1 -- 172.21.15.181:0/3934850079 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949002ccb0 con 0x7f94980b2740 2024-03-29T04:09:36.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67997d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f679415c0e0 0x7f67940cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f40797f3640 1 Processor -- start 2024-03-29T04:09:36.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67997d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 0x7f679415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67997d6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67940cdc40 con 0x7f67940b44d0 2024-03-29T04:09:36.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67997d6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f67940cd270 con 0x7f679415c0e0 2024-03-29T04:09:36.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67997d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67940b5a40 con 0x7f67940cf7a0 2024-03-29T04:09:36.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67937fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f67940b44d0 0x7f67940cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:35120/0 (socket says 172.21.15.181:35120) 2024-03-29T04:09:36.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f6798fd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 0x7f679415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f67937fe640 1 -- 172.21.15.181:0/3605890820 learned_addr learned my addr 172.21.15.181:0/3605890820 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.845+0000 7f6793fff640 1 --2- 172.21.15.181:0/3605890820 >> v2:172.21.15.187:3300/0 conn(0x7f679415c0e0 0x7f67940cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f6798fd5640 1 -- 172.21.15.181:0/3605890820 >> v2:172.21.15.187:3300/0 conn(0x7f679415c0e0 msgr2=0x7f67940cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f6798fd5640 1 --2- 172.21.15.181:0/3605890820 >> v2:172.21.15.187:3300/0 conn(0x7f679415c0e0 0x7f67940cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f6798fd5640 1 -- 172.21.15.181:0/3605890820 >> v1:172.21.15.187:6789/0 conn(0x7f67940b44d0 legacy=0x7f67940cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f6792ffd640 1 -- 172.21.15.181:0/3605890820 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 871611512 0 0) 0x7f67940cdc40 con 0x7f67940b44d0 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f6798fd5640 1 -- 172.21.15.181:0/3605890820 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67940b4a90 con 0x7f67940cf7a0 2024-03-29T04:09:36.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f40797f3640 1 -- start start 2024-03-29T04:09:36.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f948ffff640 1 -- 172.21.15.181:0/3934850079 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949002ce60 con 0x7f94980b2740 2024-03-29T04:09:36.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f949f2d9640 1 -- 172.21.15.181:0/3934850079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94980b2740 msgr2=0x7f949815a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3934850079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94980b2740 0x7f949815a4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9498071080 tx=0x7f949002bf70 comp rx=0 tx=0).stop 2024-03-29T04:09:36.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f0ad4d31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 0x7f0ad00cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f0ad4d31640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad00cb680 con 0x7f0ad00ceb90 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f0ad4d31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad006f9c0 con 0x7f0ad006f3a0 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.846+0000 7f0ad4d31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad006b790 con 0x7f0ad015a9e0 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f949f2d9640 1 -- 172.21.15.181:0/3934850079 shutdown_connections 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3934850079 >> v2:172.21.15.181:3300/0 conn(0x7f949815f000 0x7f94980cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3934850079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94980b2740 0x7f949815a4b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f0acdd9b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0ad00ceb90 0x7f0ad006ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57154/0 (socket says 172.21.15.181:57154) 2024-03-29T04:09:36.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f0acdd9b640 1 -- 172.21.15.181:0/1727858740 learned_addr learned my addr 172.21.15.181:0/1727858740 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:36.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f949f2d9640 1 -- 172.21.15.181:0/3934850079 >> 172.21.15.181:0/3934850079 conn(0x7f94980b1c20 msgr2=0x7f94980b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f6798fd5640 1 --2- 172.21.15.181:0/3605890820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 0x7f679415c4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67840097a0 tx=0x7f678402fd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3513ccb8797c5093 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f0ace59c640 1 --2- 172.21.15.181:0/1727858740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 0x7f0ad00cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f949f2d9640 1 -- 172.21.15.181:0/3934850079 shutdown_connections 2024-03-29T04:09:36.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f2232250640 1 -- 172.21.15.181:0/1600371000 shutdown_connections 2024-03-29T04:09:36.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f0acd59a640 1 -- 172.21.15.181:0/1727858740 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1640658863 0 0) 0x7f0ad00cb680 con 0x7f0ad00ceb90 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f2232250640 1 -- 172.21.15.181:0/1600371000 wait complete. 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f406c00aa90 0x7f406c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 0x7f406c0494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f406c052cb0 con 0x7f406c00aa90 2024-03-29T04:09:36.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f406c052b30 con 0x7f406c0df380 2024-03-29T04:09:36.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f0acd59a640 1 -- 172.21.15.181:0/1727858740 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ab4003520 con 0x7f0ad00ceb90 2024-03-29T04:09:36.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f0ace59c640 1 -- 172.21.15.181:0/1727858740 >> v1:172.21.15.187:6789/0 conn(0x7f0ad006f3a0 legacy=0x7f0ad006c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f0ace59c640 1 -- 172.21.15.181:0/1727858740 >> v1:172.21.15.181:6789/0 conn(0x7f0ad00ceb90 legacy=0x7f0ad006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f0ace59c640 1 -- 172.21.15.181:0/1727858740 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad006ba70 con 0x7f0ad015a9e0 2024-03-29T04:09:36.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f40797f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0382b0 con 0x7f406c050190 2024-03-29T04:09:36.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f4078ff2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f40737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 0x7f406c0494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f4073fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f406c00aa90 0x7f406c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f40737fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 0x7f406c0494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51390/0 (socket says 172.21.15.181:51390) 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f6792ffd640 1 -- 172.21.15.181:0/3605890820 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678403d070 con 0x7f67940cf7a0 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f6792ffd640 1 -- 172.21.15.181:0/3605890820 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784030ab0 con 0x7f67940cf7a0 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.847+0000 7f6792ffd640 1 -- 172.21.15.181:0/3605890820 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784038440 con 0x7f67940cf7a0 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 -- 172.21.15.181:0/3605890820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 msgr2=0x7f679415c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 --2- 172.21.15.181:0/3605890820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 0x7f679415c4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67840097a0 tx=0x7f678402fd00 comp rx=0 tx=0).stop 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f40737fe640 1 -- 172.21.15.181:0/361035654 learned_addr learned my addr 172.21.15.181:0/361035654 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 -- 172.21.15.181:0/3605890820 shutdown_connections 2024-03-29T04:09:36.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 --2- 172.21.15.181:0/3605890820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940cf7a0 0x7f679415c4c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 --2- 172.21.15.181:0/3605890820 >> v2:172.21.15.187:3300/0 conn(0x7f679415c0e0 0x7f67940cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f67997d6640 1 -- 172.21.15.181:0/3605890820 >> 172.21.15.181:0/3605890820 conn(0x7f67940d06c0 msgr2=0x7f67940d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 -- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 msgr2=0x7f406c0494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.848+0000 7f949f2d9640 1 -- 172.21.15.181:0/3934850079 wait complete. 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 0x7f406c0494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 -- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f406c00aa90 msgr2=0x7f406c0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f406c00aa90 0x7f406c0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 -- 172.21.15.181:0/361035654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c04a430 con 0x7f406c0df380 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f0ace59c640 1 --2- 172.21.15.181:0/1727858740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 0x7f0ad00cb140 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009dc0 tx=0x7f0ac802bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c928c5ba031c0af server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f67997d6640 1 -- 172.21.15.181:0/3605890820 shutdown_connections 2024-03-29T04:09:36.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f2232250640 1 Processor -- start 2024-03-29T04:09:36.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f67997d6640 1 -- 172.21.15.181:0/3605890820 wait complete. 2024-03-29T04:09:36.847 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f0acd59a640 1 -- 172.21.15.181:0/1727858740 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8039070 con 0x7f0ad015a9e0 2024-03-29T04:09:36.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f0acd59a640 1 -- 172.21.15.181:0/1727858740 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac802ccc0 con 0x7f0ad015a9e0 2024-03-29T04:09:36.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f4078ff2640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0499f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4060032000 tx=0x7f40600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f0acd59a640 1 -- 172.21.15.181:0/1727858740 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8034660 con 0x7f0ad015a9e0 2024-03-29T04:09:36.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7f2232250640 1 -- start start 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060042070 con 0x7f406c0df380 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f949f2d9640 1 Processor -- start 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f2232250640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f2232250640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406003d850 con 0x7f406c0df380 2024-03-29T04:09:36.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060037040 con 0x7f406c0df380 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c04a710 con 0x7f406c0df380 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c0dfcc0 con 0x7f406c0df380 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f67997d6640 1 Processor -- start 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7fcf9f574640 1 Processor -- start 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.849+0000 7fcf9f574640 1 -- start start 2024-03-29T04:09:36.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9f574640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f2230a4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f67997d6640 1 -- start start 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9f574640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf900544b0 con 0x7fcf90050190 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9f574640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf900541d0 con 0x7fcf900df210 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9f574640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf90005f20 con 0x7fcf90053c60 2024-03-29T04:09:36.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51404/0 (socket says 172.21.15.181:51404) 2024-03-29T04:09:36.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 -- 172.21.15.181:0/2288323872 learned_addr learned my addr 172.21.15.181:0/2288323872 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f2230a4d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c0b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51394/0 (socket says 172.21.15.181:51394) 2024-03-29T04:09:36.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.850+0000 7f949f2d9640 1 -- start start 2024-03-29T04:09:36.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94980b2740 0x7f949806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949815f000 0x7f949814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f94980ccd30 con 0x7f949815a9f0 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94980cb1e0 con 0x7f949815f000 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f949f2d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94980cceb0 con 0x7f94980b2740 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c0089c0 con 0x7f406c0df380 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7f223124e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f223124e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34650/0 (socket says 172.21.15.181:34650) 2024-03-29T04:09:36.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f2232250640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f222c0ce4d0 0x7f222c16e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f2232250640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f222c0ce960 con 0x7f222c086e10 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f2232250640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f222c0cf7f0 con 0x7f222c0ce4d0 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f2232250640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f222c0ceae0 con 0x7f222c0b17b0 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 -- 172.21.15.181:0/2288323872 >> v1:172.21.15.187:6789/0 conn(0x7fcf900df210 legacy=0x7fcf9004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 -- 172.21.15.181:0/2288323872 >> v1:172.21.15.181:6789/0 conn(0x7fcf90050190 legacy=0x7fcf90006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.851+0000 7fcf9db11640 1 -- 172.21.15.181:0/2288323872 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90006200 con 0x7fcf90053c60 2024-03-29T04:09:36.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f949d075640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94980b2740 0x7f949806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f2230a4d640 1 -- 172.21.15.181:0/3383096083 learned_addr learned my addr 172.21.15.181:0/3383096083 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f949c874640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f949c874640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34662/0 (socket says 172.21.15.181:34662) 2024-03-29T04:09:36.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f949c874640 1 -- 172.21.15.181:0/3146344838 learned_addr learned my addr 172.21.15.181:0/3146344838 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f949d876640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949815f000 0x7f949814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 0x7f679406df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679415c0e0 0x7f679415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67940ccf00 con 0x7f67940cf7a0 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67940ccc20 con 0x7f679415c0e0 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f6793fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.852+0000 7f67997d6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67940cfc10 con 0x7f67940b44d0 2024-03-29T04:09:36.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 -- 172.21.15.181:0/1727858740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 msgr2=0x7f0ad00cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/1727858740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 0x7f0ad00cb140 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009dc0 tx=0x7f0ac802bf10 comp rx=0 tx=0).stop 2024-03-29T04:09:36.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 -- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94980b2740 msgr2=0x7f949806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94980b2740 0x7f949806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 -- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 msgr2=0x7f222c0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 -- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f222c0ce4d0 msgr2=0x7f222c16e900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f222c0ce4d0 0x7f222c16e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 -- 172.21.15.181:0/3383096083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c0c8060 con 0x7f222c086e10 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f6793fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51420/0 (socket says 172.21.15.181:51420) 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f6793fff640 1 -- 172.21.15.181:0/3820263239 learned_addr learned my addr 172.21.15.181:0/3820263239 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 0x7f679406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 -- 172.21.15.181:0/1727858740 shutdown_connections 2024-03-29T04:09:36.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f6798fd5640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679415c0e0 0x7f679415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 -- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949815f000 msgr2=0x7f949814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/1727858740 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad015a9e0 0x7f0ad00cb140 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009dc0 tx=0x7f0ac802bf10 comp rx=0 tx=0).stop 2024-03-29T04:09:36.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 -- 172.21.15.181:0/1727858740 >> 172.21.15.181:0/1727858740 conn(0x7f0ad00cd250 msgr2=0x7f0ad00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949815f000 0x7f949814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 -- 172.21.15.181:0/1727858740 shutdown_connections 2024-03-29T04:09:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f223124e640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f221c036000 tx=0x7f221c030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 -- 172.21.15.181:0/3146344838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949814e2b0 con 0x7f949815a9f0 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f949c874640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f948800c480 tx=0x7f948800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 -- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 msgr2=0x7f67941482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 -- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679415c0e0 msgr2=0x7f679415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679415c0e0 0x7f679415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f67937fe640 1 -- 172.21.15.181:0/3820263239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679406f930 con 0x7f67940cf7a0 2024-03-29T04:09:36.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94880052d0 con 0x7f949815a9f0 2024-03-29T04:09:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f6793fff640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4060045070 con 0x7f406c0df380 2024-03-29T04:09:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f4070ff9640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 0x7f405806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f40600ad3e0 con 0x7f406c0df380 2024-03-29T04:09:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c008e10 con 0x7f222c086e10 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f4073fff640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 0x7f405806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9498142410 con 0x7f949815a9f0 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9488005470 con 0x7f949815a9f0 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488012870 con 0x7f949815a9f0 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c00a670 con 0x7f222c086e10 2024-03-29T04:09:36.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f2232250640 1 -- 172.21.15.181:0/3383096083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c0c8300 con 0x7f222c086e10 2024-03-29T04:09:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7fcf9db11640 1 --2- 172.21.15.181:0/2288323872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf9807ac00 tx=0x7fcf98069360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a84fe343898eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f67937fe640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 0x7f679406df90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f679406d150 tx=0x7f678800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f2232250640 1 -- 172.21.15.181:0/3383096083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c0c88c0 con 0x7f222c086e10 2024-03-29T04:09:36.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f4073fff640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 0x7f405806eac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40740510d0 tx=0x7f407406b4b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788017070 con 0x7f67940cf7a0 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6788005ce0 con 0x7f67940cf7a0 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788007310 con 0x7f67940cf7a0 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.855+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679406e890 con 0x7f67940cf7a0 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7fcf8ffff640 1 -- 172.21.15.181:0/2288323872 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf980b7070 con 0x7fcf90053c60 2024-03-29T04:09:36.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.855+0000 7fcf8ffff640 1 -- 172.21.15.181:0/2288323872 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf980b07d0 con 0x7fcf90053c60 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 -- 172.21.15.181:0/2288323872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 msgr2=0x7fcf90038070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 --2- 172.21.15.181:0/2288323872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf9807ac00 tx=0x7fcf98069360 comp rx=0 tx=0).stop 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.853+0000 7f0ad4d31640 1 -- 172.21.15.181:0/1727858740 wait complete. 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.855+0000 7f0ad4d31640 1 Processor -- start 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.854+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94980c71c0 con 0x7f949815a9f0 2024-03-29T04:09:36.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c00aa60 con 0x7f222c086e10 2024-03-29T04:09:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 -- 172.21.15.181:0/2288323872 shutdown_connections 2024-03-29T04:09:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 --2- 172.21.15.181:0/2288323872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90053c60 0x7fcf90038070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 -- 172.21.15.181:0/2288323872 >> 172.21.15.181:0/2288323872 conn(0x7fcf90009fa0 msgr2=0x7fcf9000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.855+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679406ee50 con 0x7f67940cf7a0 2024-03-29T04:09:36.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949816cc60 con 0x7f949815a9f0 2024-03-29T04:09:36.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c075890 con 0x7f222c086e10 2024-03-29T04:09:36.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.855+0000 7f0ad4d31640 1 -- start start 2024-03-29T04:09:36.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f0ad4d31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 0x7f0ad02019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67940749e0 con 0x7f67940cf7a0 2024-03-29T04:09:36.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f0ad4d31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 0x7f0ad02008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7f0ad4d31640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 0x7f0ad0155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0ace59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 0x7f0ad02019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0ace59c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 0x7f0ad02019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:51422/0 (socket says 172.21.15.181:51422) 2024-03-29T04:09:36.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0acdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 0x7f0ad02008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0ad4d31640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00cf170 con 0x7f0ad0155700 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0ad4d31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00ceff0 con 0x7f0ad00ceb90 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.856+0000 7fcf9f574640 1 -- 172.21.15.181:0/2288323872 shutdown_connections 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7fcf9f574640 1 -- 172.21.15.181:0/2288323872 wait complete. 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0acdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 0x7f0ad02008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54250/0 (socket says 172.21.15.181:54250) 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f0acdd9b640 1 -- 172.21.15.181:0/3508012478 learned_addr learned my addr 172.21.15.181:0/3508012478 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad00cf2f0 con 0x7f0ad006f3a0 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 0x7f0ad0155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.857+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f678801a070 con 0x7f67940cf7a0 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 -- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 msgr2=0x7f0ad02019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 0x7f0ad02019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 -- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 msgr2=0x7f0ad02008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 0x7f0ad02008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f0aced9d640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad0201f30 con 0x7f0ad0155700 2024-03-29T04:09:36.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f221c049020 con 0x7f222c086e10 2024-03-29T04:09:36.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9488012a10 con 0x7f949815a9f0 2024-03-29T04:09:36.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7f6790ff9640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 0x7f678006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f678808ac70 con 0x7f67940cf7a0 2024-03-29T04:09:36.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f0aced9d640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 0x7f0ad0155b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c3c0 tx=0x7f0abc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f2221ffb640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 0x7f221006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f6793fff640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 0x7f678006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7fcf9f574640 1 Processor -- start 2024-03-29T04:09:36.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.858+0000 7fcf9f574640 1 -- start start 2024-03-29T04:09:36.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f948dffb640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 0x7f947c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f221c0adc50 con 0x7f222c086e10 2024-03-29T04:09:36.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f948808c110 con 0x7f949815a9f0 2024-03-29T04:09:36.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7f949d075640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 0x7f947c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7fcf9f574640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90050190 0x7fcf90049260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7fcf9f574640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.859+0000 7fcf9f574640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 0x7fcf900d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9f574640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90037a80 con 0x7fcf900df210 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9f574640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90037900 con 0x7fcf90053c60 2024-03-29T04:09:36.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9f574640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf900df5f0 con 0x7fcf90050190 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54260/0 (socket says 172.21.15.181:54260) 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 -- 172.21.15.181:0/1566200034 learned_addr learned my addr 172.21.15.181:0/1566200034 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9db11640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 0x7fcf900d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7f2230a4d640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 0x7f221006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 -- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90050190 msgr2=0x7fcf90049260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:36.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90050190 0x7fcf90049260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 -- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 msgr2=0x7fcf900d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7f949d075640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 0x7f947c06eac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9490035000 tx=0x7f9490030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 0x7fcf900d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 -- 172.21.15.181:0/1566200034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90186b40 con 0x7fcf90053c60 2024-03-29T04:09:36.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9db11640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 0x7fcf900d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:36.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7f6793fff640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 0x7f678006eac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6794152d40 tx=0x7f678c009970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7f2230a4d640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 0x7f221006eac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f222c073660 tx=0x7f2224009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.860+0000 7fcf9cb0f640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b3f0 tx=0x7fcf8800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.862+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88015020 con 0x7fcf90053c60 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.862+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf90186e20 con 0x7fcf90053c60 2024-03-29T04:09:36.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.862+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f406007b0e0 con 0x7f406c0df380 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.862+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9004b120 con 0x7fcf90053c60 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.862+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf88011830 con 0x7fcf90053c60 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.863+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88011c20 con 0x7fcf90053c60 2024-03-29T04:09:36.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.864+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf90005380 con 0x7fcf90053c60 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.864+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf88018020 con 0x7fcf90053c60 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.864+0000 7fcf8dffb640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 0x7fcf7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.864+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf8808b460 con 0x7fcf90053c60 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.864+0000 7fcf9d310640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 0x7fcf7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.865+0000 7fcf9d310640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 0x7fcf7c06e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf94003c50 tx=0x7fcf940079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.867+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f949816cc60 con 0x7f949815a9f0 2024-03-29T04:09:36.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.873+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf8801f030 con 0x7fcf90053c60 2024-03-29T04:09:36.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.876+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f221c050020 con 0x7f222c086e10 2024-03-29T04:09:36.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.886+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0052d0 con 0x7f0ad0155700 2024-03-29T04:09:36.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.886+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc005470 con 0x7f0ad0155700 2024-03-29T04:09:36.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.886+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc007260 con 0x7f0ad0155700 2024-03-29T04:09:36.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.886+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f67940749e0 con 0x7f67940cf7a0 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.886+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad0200e00 con 0x7f0ad0155700 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.887+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad015c460 con 0x7f0ad0155700 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.889+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0abc007400 con 0x7f0ad0155700 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f0ac6ffd640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 0x7f0ab806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0abc08ad10 con 0x7f0ad0155700 2024-03-29T04:09:36.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f0ace59c640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 0x7f0ab806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.889+0000 7f29a1122640 1 Processor -- start 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.889+0000 7f29a1122640 1 -- start start 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f29a1122640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0ccd90 0x7f299c0d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f29a1122640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f299c15d2f0 0x7f299c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f29a1122640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c15dbb0 con 0x7f299c0b3d20 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f29a1122640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f299c0b46c0 con 0x7f299c15d2f0 2024-03-29T04:09:36.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.890+0000 7f29a1122640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f299c15d8d0 con 0x7f299c0ccd90 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f299c15d2f0 0x7f299c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f299c15d2f0 0x7f299c0b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:54264/0 (socket says 172.21.15.181:54264) 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 -- 172.21.15.181:0/2366433084 learned_addr learned my addr 172.21.15.181:0/2366433084 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f0ace59c640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 0x7f0ab806eac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8036000 tx=0x7f0ac8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 -- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0ccd90 msgr2=0x7f299c0d0b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T04:09:36.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 --2- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0ccd90 0x7f299c0d0b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 -- 172.21.15.181:0/2366433084 >> v1:172.21.15.181:6789/0 conn(0x7f299c0b3d20 legacy=0x7f299c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:36.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.891+0000 7f299b59e640 1 -- 172.21.15.181:0/2366433084 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0d0090 con 0x7f299c15d2f0 2024-03-29T04:09:36.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.892+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad0075900 con 0x7f0ad0155700 2024-03-29T04:09:36.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.892+0000 7f299b59e640 1 --2- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c15d2f0 0x7f299c0b4180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2994009e50 tx=0x7f299402bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b2190eb04e57c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.892+0000 7f2999d9b640 1 -- 172.21.15.181:0/2366433084 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299402caa0 con 0x7f299c15d2f0 2024-03-29T04:09:36.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.893+0000 7f2999d9b640 1 -- 172.21.15.181:0/2366433084 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299402cc40 con 0x7f299c15d2f0 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 -- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c15d2f0 msgr2=0x7f299c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 --2- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c15d2f0 0x7f299c0b4180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2994009e50 tx=0x7f299402bf00 comp rx=0 tx=0).stop 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 -- 172.21.15.181:0/2366433084 shutdown_connections 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 --2- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c15d2f0 0x7f299c0b4180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 --2- 172.21.15.181:0/2366433084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0ccd90 0x7f299c0d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 -- 172.21.15.181:0/2366433084 >> 172.21.15.181:0/2366433084 conn(0x7f299c0672c0 msgr2=0x7f299c0676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 -- 172.21.15.181:0/2366433084 shutdown_connections 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.894+0000 7f29a1122640 1 -- 172.21.15.181:0/2366433084 wait complete. 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.897+0000 7f29a1122640 1 Processor -- start 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.897+0000 7f29a1122640 1 -- start start 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0b3d20 0x7f299c1e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c0ccd90 0x7f299c1e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0d10b0 con 0x7f299c15d2f0 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0d0650 con 0x7f299c0ccd90 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f29a1122640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0cd170 con 0x7f299c0b3d20 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f299b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f299b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34720/0 (socket says 172.21.15.181:34720) 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f299b59e640 1 -- 172.21.15.181:0/762730779 learned_addr learned my addr 172.21.15.181:0/762730779 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.898+0000 7f299ad9d640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0b3d20 0x7f299c1e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 -- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0b3d20 msgr2=0x7f299c1e4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0b3d20 0x7f299c1e4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 -- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c0ccd90 msgr2=0x7f299c1e53b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c0ccd90 0x7f299c1e53b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 -- 172.21.15.181:0/762730779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c1e70d0 con 0x7f299c15d2f0 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.899+0000 7f299b59e640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2994009e50 tx=0x7f2994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:36.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.902+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299404a070 con 0x7f299c15d2f0 2024-03-29T04:09:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.902+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0ad0200e00 con 0x7f0ad0155700 2024-03-29T04:09:36.893 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.902+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c1e73b0 con 0x7f299c15d2f0 2024-03-29T04:09:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.902+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1e6d80 con 0x7f299c15d2f0 2024-03-29T04:09:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.902+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2994046a50 con 0x7f299c15d2f0 2024-03-29T04:09:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.903+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994046e40 con 0x7f299c15d2f0 2024-03-29T04:09:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.906+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f299403c3d0 con 0x7f299c15d2f0 2024-03-29T04:09:36.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.906+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c005290 con 0x7f299c15d2f0 2024-03-29T04:09:36.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.916+0000 7f29837fe640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 0x7f298806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:36.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.917+0000 7f299ad9d640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 0x7f298806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:36.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.917+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f29940b6530 con 0x7f299c15d2f0 2024-03-29T04:09:36.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.917+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f29940b8a40 con 0x7f299c15d2f0 2024-03-29T04:09:36.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:36.918+0000 7f299ad9d640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 0x7f298806eac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.194+0000 7f28e759e640 1 -- 172.21.15.181:0/2488547349 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f28cc005850 con 0x7f28e814c7c0 2024-03-29T04:09:37.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.194+0000 7f28d6ffd640 1 -- 172.21.15.181:0/2488547349 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28e005d910 con 0x7f28e814c7c0 2024-03-29T04:09:37.174 INFO:teuthology.orchestra.run.smithi181.stdout:55834574859 2024-03-29T04:09:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.197+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 msgr2=0x7f28d006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.197+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 0x7f28d006ec10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b2790 tx=0x7f28dc009040 comp rx=0 tx=0).stop 2024-03-29T04:09:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.198+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 msgr2=0x7f28e8144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.198+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28e0007d40 tx=0x7f28e000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 shutdown_connections 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28d006c750 0x7f28d006ec10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28e814c7c0 0x7f28e8144c40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e80b2950 0x7f28e814c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 --2- 172.21.15.181:0/2488547349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e8166420 0x7f28e814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 >> 172.21.15.181:0/2488547349 conn(0x7f28e80cc7b0 msgr2=0x7f28e8164970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 shutdown_connections 2024-03-29T04:09:37.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.202+0000 7f28d4ff9640 1 -- 172.21.15.181:0/2488547349 wait complete. 2024-03-29T04:09:37.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-03-29T04:09:37.199 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.253+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f679415c680 con 0x7f67940cf7a0 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.253+0000 7f6790ff9640 1 -- 172.21.15.181:0/3820263239 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6788058970 con 0x7f67940cf7a0 2024-03-29T04:09:37.233 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 msgr2=0x7f678006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 0x7f678006eac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6794152d40 tx=0x7f678c009970 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 msgr2=0x7f679406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 0x7f679406df90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f679406d150 tx=0x7f678800c7f0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 shutdown_connections 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f679415c0e0 0x7f679415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67940cf7a0 0x7f679406df90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f678006c600 0x7f678006eac0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6794152d40 tx=0x7f678c009970 comp rx=0 tx=0).stop 2024-03-29T04:09:37.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 --2- 172.21.15.181:0/3820263239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67940b44d0 0x7f67941482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.255+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 >> 172.21.15.181:0/3820263239 conn(0x7f67940d06c0 msgr2=0x7f679415ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.256+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 shutdown_connections 2024-03-29T04:09:37.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.256+0000 7f67997d6640 1 -- 172.21.15.181:0/3820263239 wait complete. 2024-03-29T04:09:37.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-03-29T04:09:37.246 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.271+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f406c005380 con 0x7f406c0df380 2024-03-29T04:09:37.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.271+0000 7f4070ff9640 1 -- 172.21.15.181:0/361035654 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f406007f3e0 con 0x7f406c0df380 2024-03-29T04:09:37.251 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.273+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 msgr2=0x7f405806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.273+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 0x7f405806eac0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40740510d0 tx=0x7f407406b4b0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.273+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 msgr2=0x7f406c0499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.273+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0499f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4060032000 tx=0x7f40600046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.273+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 shutdown_connections 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c0df380 0x7f406c0499f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c600 0x7f405806eac0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c050190 0x7f406c0494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 --2- 172.21.15.181:0/361035654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f406c00aa90 0x7f406c0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 >> 172.21.15.181:0/361035654 conn(0x7f406c009fa0 msgr2=0x7f406c054240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 shutdown_connections 2024-03-29T04:09:37.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.274+0000 7f40797f3640 1 -- 172.21.15.181:0/361035654 wait complete. 2024-03-29T04:09:37.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-03-29T04:09:37.265 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.286+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcf90052bd0 con 0x7fcf90053c60 2024-03-29T04:09:37.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.287+0000 7fcf8dffb640 1 -- 172.21.15.181:0/1566200034 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf880597d0 con 0x7fcf90053c60 2024-03-29T04:09:37.266 INFO:teuthology.orchestra.run.smithi181.stdout:55834574857 2024-03-29T04:09:37.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 msgr2=0x7fcf7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 0x7fcf7c06e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf94003c50 tx=0x7fcf940079b0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 msgr2=0x7fcf900d4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b3f0 tx=0x7fcf8800b8b0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 shutdown_connections 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf900df210 0x7fcf900d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf90053c60 0x7fcf900d4aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf7c06c530 0x7fcf7c06e9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 --2- 172.21.15.181:0/1566200034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf90050190 0x7fcf90049260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 >> 172.21.15.181:0/1566200034 conn(0x7fcf90009fa0 msgr2=0x7fcf900db0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 shutdown_connections 2024-03-29T04:09:37.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.290+0000 7fcf9f574640 1 -- 172.21.15.181:0/1566200034 wait complete. 2024-03-29T04:09:37.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.293+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9498075900 con 0x7f949815a9f0 2024-03-29T04:09:37.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.293+0000 7f948dffb640 1 -- 172.21.15.181:0/3146344838 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f948805a460 con 0x7f949815a9f0 2024-03-29T04:09:37.273 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 msgr2=0x7f947c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 0x7f947c06eac0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9490035000 tx=0x7f9490030000 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 msgr2=0x7f949814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f948800c480 tx=0x7f948800c940 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 shutdown_connections 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949815f000 0x7f949814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f949815a9f0 0x7f949814d830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f947c06c600 0x7f947c06eac0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 --2- 172.21.15.181:0/3146344838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94980b2740 0x7f949806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 >> 172.21.15.181:0/3146344838 conn(0x7f94980b1c20 msgr2=0x7f94980ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.296+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 shutdown_connections 2024-03-29T04:09:37.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.297+0000 7f949f2d9640 1 -- 172.21.15.181:0/3146344838 wait complete. 2024-03-29T04:09:37.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.297+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f222c0cf360 con 0x7f222c086e10 2024-03-29T04:09:37.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.297+0000 7f2221ffb640 1 -- 172.21.15.181:0/3383096083 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f221c07bfa0 con 0x7f222c086e10 2024-03-29T04:09:37.278 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.300+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 msgr2=0x7f221006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.300+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 0x7f221006eac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f222c073660 tx=0x7f2224009040 comp rx=0 tx=0).stop 2024-03-29T04:09:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.300+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 msgr2=0x7f222c0c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.300+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f221c036000 tx=0x7f221c030070 comp rx=0 tx=0).stop 2024-03-29T04:09:37.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 shutdown_connections 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f222c0ce4d0 0x7f222c16e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f221006c600 0x7f221006eac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f222c0b17b0 0x7f222c0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 --2- 172.21.15.181:0/3383096083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f222c086e10 0x7f222c0c5840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 >> 172.21.15.181:0/3383096083 conn(0x7f222c0cc7b0 msgr2=0x7f222c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 shutdown_connections 2024-03-29T04:09:37.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.301+0000 7f22177fe640 1 -- 172.21.15.181:0/3383096083 wait complete. 2024-03-29T04:09:37.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.303+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0ad00b0550 con 0x7f0ad0155700 2024-03-29T04:09:37.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.303+0000 7f0ac6ffd640 1 -- 172.21.15.181:0/3508012478 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ad0201f30 con 0x7f0ad0155700 2024-03-29T04:09:37.283 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-03-29T04:09:37.287 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 msgr2=0x7f0ab806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 0x7f0ab806eac0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8036000 tx=0x7f0ac8030000 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 msgr2=0x7f0ad0155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 0x7f0ad0155b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c3c0 tx=0x7f0abc00c880 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 shutdown_connections 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0155700 0x7f0ad0155b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad00ceb90 0x7f0ad02008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ab806c600 0x7f0ab806eac0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 --2- 172.21.15.181:0/3508012478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad006f3a0 0x7f0ad02019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 >> 172.21.15.181:0/3508012478 conn(0x7f0ad00cd250 msgr2=0x7f0ad00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 shutdown_connections 2024-03-29T04:09:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.307+0000 7f0ad4d31640 1 -- 172.21.15.181:0/3508012478 wait complete. 2024-03-29T04:09:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.309+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f297c005580 con 0x7f299c15d2f0 2024-03-29T04:09:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.309+0000 7f29837fe640 1 -- 172.21.15.181:0/762730779 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2994084880 con 0x7f299c15d2f0 2024-03-29T04:09:37.289 INFO:teuthology.orchestra.run.smithi181.stdout:55834574858 2024-03-29T04:09:37.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 msgr2=0x7f298806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 0x7f298806eac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2990006fd0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 msgr2=0x7f299c15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2994009e50 tx=0x7f2994004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 shutdown_connections 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299c15d2f0 0x7f299c15e950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f299c0ccd90 0x7f299c1e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f298806c600 0x7f298806eac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 --2- 172.21.15.181:0/762730779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f299c0b3d20 0x7f299c1e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 >> 172.21.15.181:0/762730779 conn(0x7f299c0672c0 msgr2=0x7f299c156730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 shutdown_connections 2024-03-29T04:09:37.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.313+0000 7f29a1122640 1 -- 172.21.15.181:0/762730779 wait complete. 2024-03-29T04:09:37.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-03-29T04:09:37.294 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-03-29T04:09:37.297 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-03-29T04:09:37.304 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-03-29T04:09:37.307 DEBUG:teuthology.parallel:result is None 2024-03-29T04:09:37.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:09:37.307 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:37.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.466+0000 7fadd32d9640 1 Processor -- start 2024-03-29T04:09:37.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.466+0000 7fadd32d9640 1 -- start start 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd32d9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadcc141f20 0x7fadcc0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd32d9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadcc142880 con 0x7fadcc15cc90 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd32d9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadcc071de0 con 0x7fadcc141a90 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd32d9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fadcc0cef30 con 0x7fadcc141f20 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd1075640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fadcc15cc90 0x7fadcc15d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57194/0 (socket says 172.21.15.181:57194) 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd1075640 1 -- 172.21.15.181:0/152949958 learned_addr learned my addr 172.21.15.181:0/152949958 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.467+0000 7fadd1876640 1 --2- 172.21.15.181:0/152949958 >> v2:172.21.15.187:3300/0 conn(0x7fadcc141f20 0x7fadcc0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829712561 0 0) 0x7fadcc142880 con 0x7fadcc15cc90 2024-03-29T04:09:37.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb8003520 con 0x7fadcc15cc90 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478854554 0 0) 0x7fadcc071de0 con 0x7fadcc141a90 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadcc142880 con 0x7fadcc141a90 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadd1876640 1 -- 172.21.15.181:0/152949958 >> v1:172.21.15.187:6789/0 conn(0x7fadcc141a90 legacy=0x7fadcc0cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadd1876640 1 -- 172.21.15.181:0/152949958 >> v1:172.21.15.181:6789/0 conn(0x7fadcc15cc90 legacy=0x7fadcc15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadd1876640 1 -- 172.21.15.181:0/152949958 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc0720c0 con 0x7fadcc141f20 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.468+0000 7fadd1876640 1 --2- 172.21.15.181:0/152949958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0cfab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadc40097a0 tx=0x7fadc402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356adcfe9ffad728 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4034070 con 0x7fadcc141f20 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc4030610 con 0x7fadcc141f20 2024-03-29T04:09:37.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadc3fff640 1 -- 172.21.15.181:0/152949958 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc40309e0 con 0x7fadcc141f20 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 -- 172.21.15.181:0/152949958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 msgr2=0x7fadcc0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 --2- 172.21.15.181:0/152949958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0cfab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadc40097a0 tx=0x7fadc402b960 comp rx=0 tx=0).stop 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 -- 172.21.15.181:0/152949958 shutdown_connections 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 --2- 172.21.15.181:0/152949958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0cfab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 -- 172.21.15.181:0/152949958 >> 172.21.15.181:0/152949958 conn(0x7fadcc06b580 msgr2=0x7fadcc06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 -- 172.21.15.181:0/152949958 shutdown_connections 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.469+0000 7fadd32d9640 1 -- 172.21.15.181:0/152949958 wait complete. 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.470+0000 7fadd32d9640 1 Processor -- start 2024-03-29T04:09:37.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.470+0000 7fadd32d9640 1 -- start start 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc141a90 0x7fadcc1e7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc072680 con 0x7fadcc15cc90 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc072960 con 0x7fadcc141f20 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd32d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadcc142370 con 0x7fadcc141a90 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34728/0 (socket says 172.21.15.181:34728) 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 -- 172.21.15.181:0/836152585 learned_addr learned my addr 172.21.15.181:0/836152585 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd0874640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1075640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc141a90 0x7fadcc1e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 -- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc141a90 msgr2=0x7fadcc1e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc141a90 0x7fadcc1e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 -- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 msgr2=0x7fadcc0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.471+0000 7fadd1876640 1 -- 172.21.15.181:0/836152585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc1438a0 con 0x7fadcc15cc90 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadd1876640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadc402bf70 tx=0x7fadc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4034040 con 0x7fadcc15cc90 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc4030410 con 0x7fadcc15cc90 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc143b10 con 0x7fadcc15cc90 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4006e10 con 0x7fadcc15cc90 2024-03-29T04:09:37.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.472+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc1440d0 con 0x7fadcc15cc90 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.473+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc1442d0 con 0x7fadcc15cc90 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.473+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fadc4037070 con 0x7fadcc15cc90 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.474+0000 7fadc1ffb640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 0x7fadb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.474+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fadc40ad7e0 con 0x7fadcc15cc90 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.474+0000 7fadd1075640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 0x7fadb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.474+0000 7fadd1075640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 0x7fadb006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc80012e0 tx=0x7fadc8001a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.478+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fadcc1442d0 con 0x7fadcc15cc90 2024-03-29T04:09:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.651+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadcc156b50 con 0x7fadb006c530 2024-03-29T04:09:37.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.658+0000 7fadc1ffb640 1 -- 172.21.15.181:0/836152585 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fadcc156b50 con 0x7fadb006c530 2024-03-29T04:09:37.641 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:37.643 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:09:37.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 msgr2=0x7fadb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 0x7fadb006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc80012e0 tx=0x7fadc8001a30 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 msgr2=0x7fadcc15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadc402bf70 tx=0x7fadc40046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 shutdown_connections 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadcc15cc90 0x7fadcc15ec90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadcc141f20 0x7fadcc0b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fadb006c530 0x7fadb006e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 --2- 172.21.15.181:0/836152585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadcc141a90 0x7fadcc1e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 >> 172.21.15.181:0/836152585 conn(0x7fadcc06b580 msgr2=0x7fadcc074bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 shutdown_connections 2024-03-29T04:09:37.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.661+0000 7fadd32d9640 1 -- 172.21.15.181:0/836152585 wait complete. 2024-03-29T04:09:37.652 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-29T04:09:37.578328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656335+0000","last_change":"2024-03-29T04:09:15.308724+0000","last_active":"2024-03-29T04:09:22.656335+0000","last_peered":"2024-03-29T04:09:22.656335+0000","last_clean":"2024-03-29T04:09:22.656335+0000","last_became_active":"2024-03-29T04:09:15.308445+0000","last_became_peered":"2024-03-29T04:09:15.308445+0000","last_unstale":"2024-03-29T04:09:22.656335+0000","last_undegraded":"2024-03-29T04:09:22.656335+0000","last_fullsized":"2024-03-29T04:09:22.656335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329448+0000","last_change":"2024-03-29T04:09:16.307371+0000","last_active":"2024-03-29T04:09:22.329448+0000","last_peered":"2024-03-29T04:09:22.329448+0000","last_clean":"2024-03-29T04:09:22.329448+0000","last_became_active":"2024-03-29T04:09:16.307077+0000","last_became_peered":"2024-03-29T04:09:16.307077+0000","last_unstale":"2024-03-29T04:09:22.329448+0000","last_undegraded":"2024-03-29T04:09:22.329448+0000","last_fullsized":"2024-03-29T04:09:22.329448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328852+0000","last_change":"2024-03-29T04:09:20.883785+0000","last_active":"2024-03-29T04:09:22.328852+0000","last_peered":"2024-03-29T04:09:22.328852+0000","last_clean":"2024-03-29T04:09:22.328852+0000","last_became_active":"2024-03-29T04:09:20.883360+0000","last_became_peered":"2024-03-29T04:09:20.883360+0000","last_unstale":"2024-03-29T04:09:22.328852+0000","last_undegraded":"2024-03-29T04:09:22.328852+0000","last_fullsized":"2024-03-29T04:09:22.328852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.908030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329590+0000","last_change":"2024-03-29T04:09:15.301577+0000","last_active":"2024-03-29T04:09:22.329590+0000","last_peered":"2024-03-29T04:09:22.329590+0000","last_clean":"2024-03-29T04:09:22.329590+0000","last_became_active":"2024-03-29T04:09:15.301247+0000","last_became_peered":"2024-03-29T04:09:15.301247+0000","last_unstale":"2024-03-29T04:09:22.329590+0000","last_undegraded":"2024-03-29T04:09:22.329590+0000","last_fullsized":"2024-03-29T04:09:22.329590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656042+0000","last_change":"2024-03-29T04:09:16.319168+0000","last_active":"2024-03-29T04:09:22.656042+0000","last_peered":"2024-03-29T04:09:22.656042+0000","last_clean":"2024-03-29T04:09:22.656042+0000","last_became_active":"2024-03-29T04:09:16.318673+0000","last_became_peered":"2024-03-29T04:09:16.318673+0000","last_unstale":"2024-03-29T04:09:22.656042+0000","last_undegraded":"2024-03-29T04:09:22.656042+0000","last_fullsized":"2024-03-29T04:09:22.656042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:41.049934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329629+0000","last_change":"2024-03-29T04:09:20.348184+0000","last_active":"2024-03-29T04:09:22.329629+0000","last_peered":"2024-03-29T04:09:22.329629+0000","last_clean":"2024-03-29T04:09:22.329629+0000","last_became_active":"2024-03-29T04:09:20.347504+0000","last_became_peered":"2024-03-29T04:09:20.347504+0000","last_unstale":"2024-03-29T04:09:22.329629+0000","last_undegraded":"2024-03-29T04:09:22.329629+0000","last_fullsized":"2024-03-29T04:09:22.329629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656067+0000","last_change":"2024-03-29T04:09:15.308597+0000","last_active":"2024-03-29T04:09:22.656067+0000","last_peered":"2024-03-29T04:09:22.656067+0000","last_clean":"2024-03-29T04:09:22.656067+0000","last_became_active":"2024-03-29T04:09:15.308020+0000","last_became_peered":"2024-03-29T04:09:15.308020+0000","last_unstale":"2024-03-29T04:09:22.656067+0000","last_undegraded":"2024-03-29T04:09:22.656067+0000","last_fullsized":"2024-03-29T04:09:22.656067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:40.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655201+0000","last_change":"2024-03-29T04:09:16.321774+0000","last_active":"2024-03-29T04:09:22.655201+0000","last_peered":"2024-03-29T04:09:22.655201+0000","last_clean":"2024-03-29T04:09:22.655201+0000","last_became_active":"2024-03-29T04:09:16.321464+0000","last_became_peered":"2024-03-29T04:09:16.321464+0000","last_unstale":"2024-03-29T04:09:22.655201+0000","last_undegraded":"2024-03-29T04:09:22.655201+0000","last_fullsized":"2024-03-29T04:09:22.655201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330320+0000","last_change":"2024-03-29T04:09:20.346472+0000","last_active":"2024-03-29T04:09:22.330320+0000","last_peered":"2024-03-29T04:09:22.330320+0000","last_clean":"2024-03-29T04:09:22.330320+0000","last_became_active":"2024-03-29T04:09:20.346130+0000","last_became_peered":"2024-03-29T04:09:20.346130+0000","last_unstale":"2024-03-29T04:09:22.330320+0000","last_undegraded":"2024-03-29T04:09:22.330320+0000","last_fullsized":"2024-03-29T04:09:22.330320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655700+0000","last_change":"2024-03-29T04:09:15.308906+0000","last_active":"2024-03-29T04:09:22.655700+0000","last_peered":"2024-03-29T04:09:22.655700+0000","last_clean":"2024-03-29T04:09:22.655700+0000","last_became_active":"2024-03-29T04:09:15.308632+0000","last_became_peered":"2024-03-29T04:09:15.308632+0000","last_unstale":"2024-03-29T04:09:22.655700+0000","last_undegraded":"2024-03-29T04:09:22.655700+0000","last_fullsized":"2024-03-29T04:09:22.655700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.323173+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.322174+0000","last_became_peered":"2024-03-29T04:09:16.322174+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329721+0000","last_change":"2024-03-29T04:09:20.341688+0000","last_active":"2024-03-29T04:09:22.329721+0000","last_peered":"2024-03-29T04:09:22.329721+0000","last_clean":"2024-03-29T04:09:22.329721+0000","last_became_active":"2024-03-29T04:09:20.341328+0000","last_became_peered":"2024-03-29T04:09:20.341328+0000","last_unstale":"2024-03-29T04:09:22.329721+0000","last_undegraded":"2024-03-29T04:09:22.329721+0000","last_fullsized":"2024-03-29T04:09:22.329721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329443+0000","last_change":"2024-03-29T04:09:16.318229+0000","last_active":"2024-03-29T04:09:22.329443+0000","last_peered":"2024-03-29T04:09:22.329443+0000","last_clean":"2024-03-29T04:09:22.329443+0000","last_became_active":"2024-03-29T04:09:16.317967+0000","last_became_peered":"2024-03-29T04:09:16.317967+0000","last_unstale":"2024-03-29T04:09:22.329443+0000","last_undegraded":"2024-03-29T04:09:22.329443+0000","last_fullsized":"2024-03-29T04:09:22.329443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655679+0000","last_change":"2024-03-29T04:09:15.308430+0000","last_active":"2024-03-29T04:09:22.655679+0000","last_peered":"2024-03-29T04:09:22.655679+0000","last_clean":"2024-03-29T04:09:22.655679+0000","last_became_active":"2024-03-29T04:09:15.308131+0000","last_became_peered":"2024-03-29T04:09:15.308131+0000","last_unstale":"2024-03-29T04:09:22.655679+0000","last_undegraded":"2024-03-29T04:09:22.655679+0000","last_fullsized":"2024-03-29T04:09:22.655679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:36:32.401409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329454+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.329454+0000","last_peered":"2024-03-29T04:09:22.329454+0000","last_clean":"2024-03-29T04:09:22.329454+0000","last_became_active":"2024-03-29T04:09:20.347330+0000","last_became_peered":"2024-03-29T04:09:20.347330+0000","last_unstale":"2024-03-29T04:09:22.329454+0000","last_undegraded":"2024-03-29T04:09:22.329454+0000","last_fullsized":"2024-03-29T04:09:22.329454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329091+0000","last_change":"2024-03-29T04:09:16.311370+0000","last_active":"2024-03-29T04:09:22.329091+0000","last_peered":"2024-03-29T04:09:22.329091+0000","last_clean":"2024-03-29T04:09:22.329091+0000","last_became_active":"2024-03-29T04:09:16.311106+0000","last_became_peered":"2024-03-29T04:09:16.311106+0000","last_unstale":"2024-03-29T04:09:22.329091+0000","last_undegraded":"2024-03-29T04:09:22.329091+0000","last_fullsized":"2024-03-29T04:09:22.329091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656398+0000","last_change":"2024-03-29T04:09:15.306878+0000","last_active":"2024-03-29T04:09:22.656398+0000","last_peered":"2024-03-29T04:09:22.656398+0000","last_clean":"2024-03-29T04:09:22.656398+0000","last_became_active":"2024-03-29T04:09:15.306590+0000","last_became_peered":"2024-03-29T04:09:15.306590+0000","last_unstale":"2024-03-29T04:09:22.656398+0000","last_undegraded":"2024-03-29T04:09:22.656398+0000","last_fullsized":"2024-03-29T04:09:22.656398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330041+0000","last_change":"2024-03-29T04:09:20.347808+0000","last_active":"2024-03-29T04:09:22.330041+0000","last_peered":"2024-03-29T04:09:22.330041+0000","last_clean":"2024-03-29T04:09:22.330041+0000","last_became_active":"2024-03-29T04:09:20.347419+0000","last_became_peered":"2024-03-29T04:09:20.347419+0000","last_unstale":"2024-03-29T04:09:22.330041+0000","last_undegraded":"2024-03-29T04:09:22.330041+0000","last_fullsized":"2024-03-29T04:09:22.330041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:58.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328945+0000","last_change":"2024-03-29T04:09:16.309708+0000","last_active":"2024-03-29T04:09:22.328945+0000","last_peered":"2024-03-29T04:09:22.328945+0000","last_clean":"2024-03-29T04:09:22.328945+0000","last_became_active":"2024-03-29T04:09:16.309388+0000","last_became_peered":"2024-03-29T04:09:16.309388+0000","last_unstale":"2024-03-29T04:09:22.328945+0000","last_undegraded":"2024-03-29T04:09:22.328945+0000","last_fullsized":"2024-03-29T04:09:22.328945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655404+0000","last_change":"2024-03-29T04:09:15.308764+0000","last_active":"2024-03-29T04:09:22.655404+0000","last_peered":"2024-03-29T04:09:22.655404+0000","last_clean":"2024-03-29T04:09:22.655404+0000","last_became_active":"2024-03-29T04:09:15.308466+0000","last_became_peered":"2024-03-29T04:09:15.308466+0000","last_unstale":"2024-03-29T04:09:22.655404+0000","last_undegraded":"2024-03-29T04:09:22.655404+0000","last_fullsized":"2024-03-29T04:09:22.655404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657055+0000","last_change":"2024-03-29T04:09:20.883567+0000","last_active":"2024-03-29T04:09:22.657055+0000","last_peered":"2024-03-29T04:09:22.657055+0000","last_clean":"2024-03-29T04:09:22.657055+0000","last_became_active":"2024-03-29T04:09:20.883215+0000","last_became_peered":"2024-03-29T04:09:20.883215+0000","last_unstale":"2024-03-29T04:09:22.657055+0000","last_undegraded":"2024-03-29T04:09:22.657055+0000","last_fullsized":"2024-03-29T04:09:22.657055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330723+0000","last_change":"2024-03-29T04:09:16.307731+0000","last_active":"2024-03-29T04:09:22.330723+0000","last_peered":"2024-03-29T04:09:22.330723+0000","last_clean":"2024-03-29T04:09:22.330723+0000","last_became_active":"2024-03-29T04:09:16.307458+0000","last_became_peered":"2024-03-29T04:09:16.307458+0000","last_unstale":"2024-03-29T04:09:22.330723+0000","last_undegraded":"2024-03-29T04:09:22.330723+0000","last_fullsized":"2024-03-29T04:09:22.330723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:43.534030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654990+0000","last_change":"2024-03-29T04:09:15.306750+0000","last_active":"2024-03-29T04:09:22.654990+0000","last_peered":"2024-03-29T04:09:22.654990+0000","last_clean":"2024-03-29T04:09:22.654990+0000","last_became_active":"2024-03-29T04:09:15.306458+0000","last_became_peered":"2024-03-29T04:09:15.306458+0000","last_unstale":"2024-03-29T04:09:22.654990+0000","last_undegraded":"2024-03-29T04:09:22.654990+0000","last_fullsized":"2024-03-29T04:09:22.654990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330722+0000","last_change":"2024-03-29T04:09:20.346709+0000","last_active":"2024-03-29T04:09:22.330722+0000","last_peered":"2024-03-29T04:09:22.330722+0000","last_clean":"2024-03-29T04:09:22.330722+0000","last_became_active":"2024-03-29T04:09:20.346366+0000","last_became_peered":"2024-03-29T04:09:20.346366+0000","last_unstale":"2024-03-29T04:09:22.330722+0000","last_undegraded":"2024-03-29T04:09:22.330722+0000","last_fullsized":"2024-03-29T04:09:22.330722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656086+0000","last_change":"2024-03-29T04:09:16.311388+0000","last_active":"2024-03-29T04:09:22.656086+0000","last_peered":"2024-03-29T04:09:22.656086+0000","last_clean":"2024-03-29T04:09:22.656086+0000","last_became_active":"2024-03-29T04:09:16.311119+0000","last_became_peered":"2024-03-29T04:09:16.311119+0000","last_unstale":"2024-03-29T04:09:22.656086+0000","last_undegraded":"2024-03-29T04:09:22.656086+0000","last_fullsized":"2024-03-29T04:09:22.656086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330110+0000","last_change":"2024-03-29T04:09:15.301237+0000","last_active":"2024-03-29T04:09:22.330110+0000","last_peered":"2024-03-29T04:09:22.330110+0000","last_clean":"2024-03-29T04:09:22.330110+0000","last_became_active":"2024-03-29T04:09:15.300671+0000","last_became_peered":"2024-03-29T04:09:15.300671+0000","last_unstale":"2024-03-29T04:09:22.330110+0000","last_undegraded":"2024-03-29T04:09:22.330110+0000","last_fullsized":"2024-03-29T04:09:22.330110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330650+0000","last_change":"2024-03-29T04:09:20.882096+0000","last_active":"2024-03-29T04:09:22.330650+0000","last_peered":"2024-03-29T04:09:22.330650+0000","last_clean":"2024-03-29T04:09:22.330650+0000","last_became_active":"2024-03-29T04:09:20.881672+0000","last_became_peered":"2024-03-29T04:09:20.881672+0000","last_unstale":"2024-03-29T04:09:22.330650+0000","last_undegraded":"2024-03-29T04:09:22.330650+0000","last_fullsized":"2024-03-29T04:09:22.330650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329679+0000","last_change":"2024-03-29T04:09:16.308204+0000","last_active":"2024-03-29T04:09:22.329679+0000","last_peered":"2024-03-29T04:09:22.329679+0000","last_clean":"2024-03-29T04:09:22.329679+0000","last_became_active":"2024-03-29T04:09:16.307931+0000","last_became_peered":"2024-03-29T04:09:16.307931+0000","last_unstale":"2024-03-29T04:09:22.329679+0000","last_undegraded":"2024-03-29T04:09:22.329679+0000","last_fullsized":"2024-03-29T04:09:22.329679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656940+0000","last_change":"2024-03-29T04:09:15.303590+0000","last_active":"2024-03-29T04:09:22.656940+0000","last_peered":"2024-03-29T04:09:22.656940+0000","last_clean":"2024-03-29T04:09:22.656940+0000","last_became_active":"2024-03-29T04:09:15.303295+0000","last_became_peered":"2024-03-29T04:09:15.303295+0000","last_unstale":"2024-03-29T04:09:22.656940+0000","last_undegraded":"2024-03-29T04:09:22.656940+0000","last_fullsized":"2024-03-29T04:09:22.656940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328789+0000","last_change":"2024-03-29T04:09:20.884166+0000","last_active":"2024-03-29T04:09:22.328789+0000","last_peered":"2024-03-29T04:09:22.328789+0000","last_clean":"2024-03-29T04:09:22.328789+0000","last_became_active":"2024-03-29T04:09:20.883801+0000","last_became_peered":"2024-03-29T04:09:20.883801+0000","last_unstale":"2024-03-29T04:09:22.328789+0000","last_undegraded":"2024-03-29T04:09:22.328789+0000","last_fullsized":"2024-03-29T04:09:22.328789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:45.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656397+0000","last_change":"2024-03-29T04:09:16.307299+0000","last_active":"2024-03-29T04:09:22.656397+0000","last_peered":"2024-03-29T04:09:22.656397+0000","last_clean":"2024-03-29T04:09:22.656397+0000","last_became_active":"2024-03-29T04:09:16.306982+0000","last_became_peered":"2024-03-29T04:09:16.306982+0000","last_unstale":"2024-03-29T04:09:22.656397+0000","last_undegraded":"2024-03-29T04:09:22.656397+0000","last_fullsized":"2024-03-29T04:09:22.656397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656158+0000","last_change":"2024-03-29T04:09:15.308823+0000","last_active":"2024-03-29T04:09:22.656158+0000","last_peered":"2024-03-29T04:09:22.656158+0000","last_clean":"2024-03-29T04:09:22.656158+0000","last_became_active":"2024-03-29T04:09:15.308335+0000","last_became_peered":"2024-03-29T04:09:15.308335+0000","last_unstale":"2024-03-29T04:09:22.656158+0000","last_undegraded":"2024-03-29T04:09:22.656158+0000","last_fullsized":"2024-03-29T04:09:22.656158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:09.173137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655324+0000","last_change":"2024-03-29T04:09:20.882760+0000","last_active":"2024-03-29T04:09:22.655324+0000","last_peered":"2024-03-29T04:09:22.655324+0000","last_clean":"2024-03-29T04:09:22.655324+0000","last_became_active":"2024-03-29T04:09:20.881819+0000","last_became_peered":"2024-03-29T04:09:20.881819+0000","last_unstale":"2024-03-29T04:09:22.655324+0000","last_undegraded":"2024-03-29T04:09:22.655324+0000","last_fullsized":"2024-03-29T04:09:22.655324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655417+0000","last_change":"2024-03-29T04:09:16.318039+0000","last_active":"2024-03-29T04:09:22.655417+0000","last_peered":"2024-03-29T04:09:22.655417+0000","last_clean":"2024-03-29T04:09:22.655417+0000","last_became_active":"2024-03-29T04:09:16.317784+0000","last_became_peered":"2024-03-29T04:09:16.317784+0000","last_unstale":"2024-03-29T04:09:22.655417+0000","last_undegraded":"2024-03-29T04:09:22.655417+0000","last_fullsized":"2024-03-29T04:09:22.655417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656459+0000","last_change":"2024-03-29T04:09:15.302818+0000","last_active":"2024-03-29T04:09:22.656459+0000","last_peered":"2024-03-29T04:09:22.656459+0000","last_clean":"2024-03-29T04:09:22.656459+0000","last_became_active":"2024-03-29T04:09:15.302521+0000","last_became_peered":"2024-03-29T04:09:15.302521+0000","last_unstale":"2024-03-29T04:09:22.656459+0000","last_undegraded":"2024-03-29T04:09:22.656459+0000","last_fullsized":"2024-03-29T04:09:22.656459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656072+0000","last_change":"2024-03-29T04:09:20.351951+0000","last_active":"2024-03-29T04:09:22.656072+0000","last_peered":"2024-03-29T04:09:22.656072+0000","last_clean":"2024-03-29T04:09:22.656072+0000","last_became_active":"2024-03-29T04:09:20.351337+0000","last_became_peered":"2024-03-29T04:09:20.351337+0000","last_unstale":"2024-03-29T04:09:22.656072+0000","last_undegraded":"2024-03-29T04:09:22.656072+0000","last_fullsized":"2024-03-29T04:09:22.656072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330495+0000","last_change":"2024-03-29T04:09:16.316074+0000","last_active":"2024-03-29T04:09:22.330495+0000","last_peered":"2024-03-29T04:09:22.330495+0000","last_clean":"2024-03-29T04:09:22.330495+0000","last_became_active":"2024-03-29T04:09:16.315644+0000","last_became_peered":"2024-03-29T04:09:16.315644+0000","last_unstale":"2024-03-29T04:09:22.330495+0000","last_undegraded":"2024-03-29T04:09:22.330495+0000","last_fullsized":"2024-03-29T04:09:22.330495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:13.041449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655320+0000","last_change":"2024-03-29T04:09:15.307707+0000","last_active":"2024-03-29T04:09:22.655320+0000","last_peered":"2024-03-29T04:09:22.655320+0000","last_clean":"2024-03-29T04:09:22.655320+0000","last_became_active":"2024-03-29T04:09:15.307411+0000","last_became_peered":"2024-03-29T04:09:15.307411+0000","last_unstale":"2024-03-29T04:09:22.655320+0000","last_undegraded":"2024-03-29T04:09:22.655320+0000","last_fullsized":"2024-03-29T04:09:22.655320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656552+0000","last_change":"2024-03-29T04:09:20.346773+0000","last_active":"2024-03-29T04:09:22.656552+0000","last_peered":"2024-03-29T04:09:22.656552+0000","last_clean":"2024-03-29T04:09:22.656552+0000","last_became_active":"2024-03-29T04:09:20.346383+0000","last_became_peered":"2024-03-29T04:09:20.346383+0000","last_unstale":"2024-03-29T04:09:22.656552+0000","last_undegraded":"2024-03-29T04:09:22.656552+0000","last_fullsized":"2024-03-29T04:09:22.656552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.311447+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.311171+0000","last_became_peered":"2024-03-29T04:09:16.311171+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656334+0000","last_change":"2024-03-29T04:09:15.307046+0000","last_active":"2024-03-29T04:09:22.656334+0000","last_peered":"2024-03-29T04:09:22.656334+0000","last_clean":"2024-03-29T04:09:22.656334+0000","last_became_active":"2024-03-29T04:09:15.305761+0000","last_became_peered":"2024-03-29T04:09:15.305761+0000","last_unstale":"2024-03-29T04:09:22.656334+0000","last_undegraded":"2024-03-29T04:09:22.656334+0000","last_fullsized":"2024-03-29T04:09:22.656334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657098+0000","last_change":"2024-03-29T04:09:20.346732+0000","last_active":"2024-03-29T04:09:22.657098+0000","last_peered":"2024-03-29T04:09:22.657098+0000","last_clean":"2024-03-29T04:09:22.657098+0000","last_became_active":"2024-03-29T04:09:20.346121+0000","last_became_peered":"2024-03-29T04:09:20.346121+0000","last_unstale":"2024-03-29T04:09:22.657098+0000","last_undegraded":"2024-03-29T04:09:22.657098+0000","last_fullsized":"2024-03-29T04:09:22.657098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655371+0000","last_change":"2024-03-29T04:09:16.318196+0000","last_active":"2024-03-29T04:09:22.655371+0000","last_peered":"2024-03-29T04:09:22.655371+0000","last_clean":"2024-03-29T04:09:22.655371+0000","last_became_active":"2024-03-29T04:09:16.317906+0000","last_became_peered":"2024-03-29T04:09:16.317906+0000","last_unstale":"2024-03-29T04:09:22.655371+0000","last_undegraded":"2024-03-29T04:09:22.655371+0000","last_fullsized":"2024-03-29T04:09:22.655371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:36.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655070+0000","last_change":"2024-03-29T04:09:15.308083+0000","last_active":"2024-03-29T04:09:22.655070+0000","last_peered":"2024-03-29T04:09:22.655070+0000","last_clean":"2024-03-29T04:09:22.655070+0000","last_became_active":"2024-03-29T04:09:15.307813+0000","last_became_peered":"2024-03-29T04:09:15.307813+0000","last_unstale":"2024-03-29T04:09:22.655070+0000","last_undegraded":"2024-03-29T04:09:22.655070+0000","last_fullsized":"2024-03-29T04:09:22.655070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655909+0000","last_change":"2024-03-29T04:09:20.882216+0000","last_active":"2024-03-29T04:09:22.655909+0000","last_peered":"2024-03-29T04:09:22.655909+0000","last_clean":"2024-03-29T04:09:22.655909+0000","last_became_active":"2024-03-29T04:09:20.881801+0000","last_became_peered":"2024-03-29T04:09:20.881801+0000","last_unstale":"2024-03-29T04:09:22.655909+0000","last_undegraded":"2024-03-29T04:09:22.655909+0000","last_fullsized":"2024-03-29T04:09:22.655909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328920+0000","last_change":"2024-03-29T04:09:16.319621+0000","last_active":"2024-03-29T04:09:22.328920+0000","last_peered":"2024-03-29T04:09:22.328920+0000","last_clean":"2024-03-29T04:09:22.328920+0000","last_became_active":"2024-03-29T04:09:16.318978+0000","last_became_peered":"2024-03-29T04:09:16.318978+0000","last_unstale":"2024-03-29T04:09:22.328920+0000","last_undegraded":"2024-03-29T04:09:22.328920+0000","last_fullsized":"2024-03-29T04:09:22.328920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655084+0000","last_change":"2024-03-29T04:09:15.302638+0000","last_active":"2024-03-29T04:09:22.655084+0000","last_peered":"2024-03-29T04:09:22.655084+0000","last_clean":"2024-03-29T04:09:22.655084+0000","last_became_active":"2024-03-29T04:09:15.302211+0000","last_became_peered":"2024-03-29T04:09:15.302211+0000","last_unstale":"2024-03-29T04:09:22.655084+0000","last_undegraded":"2024-03-29T04:09:22.655084+0000","last_fullsized":"2024-03-29T04:09:22.655084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655145+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:09:22.655145+0000","last_peered":"2024-03-29T04:09:22.655145+0000","last_clean":"2024-03-29T04:09:22.655145+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:09:22.655145+0000","last_undegraded":"2024-03-29T04:09:22.655145+0000","last_fullsized":"2024-03-29T04:09:22.655145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329351+0000","last_change":"2024-03-29T04:09:20.883146+0000","last_active":"2024-03-29T04:09:22.329351+0000","last_peered":"2024-03-29T04:09:22.329351+0000","last_clean":"2024-03-29T04:09:22.329351+0000","last_became_active":"2024-03-29T04:09:20.882795+0000","last_became_peered":"2024-03-29T04:09:20.882795+0000","last_unstale":"2024-03-29T04:09:22.329351+0000","last_undegraded":"2024-03-29T04:09:22.329351+0000","last_fullsized":"2024-03-29T04:09:22.329351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655937+0000","last_change":"2024-03-29T04:09:16.316245+0000","last_active":"2024-03-29T04:09:22.655937+0000","last_peered":"2024-03-29T04:09:22.655937+0000","last_clean":"2024-03-29T04:09:22.655937+0000","last_became_active":"2024-03-29T04:09:16.315972+0000","last_became_peered":"2024-03-29T04:09:16.315972+0000","last_unstale":"2024-03-29T04:09:22.655937+0000","last_undegraded":"2024-03-29T04:09:22.655937+0000","last_fullsized":"2024-03-29T04:09:22.655937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656723+0000","last_change":"2024-03-29T04:09:15.304758+0000","last_active":"2024-03-29T04:09:22.656723+0000","last_peered":"2024-03-29T04:09:22.656723+0000","last_clean":"2024-03-29T04:09:22.656723+0000","last_became_active":"2024-03-29T04:09:15.304450+0000","last_became_peered":"2024-03-29T04:09:15.304450+0000","last_unstale":"2024-03-29T04:09:22.656723+0000","last_undegraded":"2024-03-29T04:09:22.656723+0000","last_fullsized":"2024-03-29T04:09:22.656723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329391+0000","last_change":"2024-03-29T04:09:20.882224+0000","last_active":"2024-03-29T04:09:22.329391+0000","last_peered":"2024-03-29T04:09:22.329391+0000","last_clean":"2024-03-29T04:09:22.329391+0000","last_became_active":"2024-03-29T04:09:20.881813+0000","last_became_peered":"2024-03-29T04:09:20.881813+0000","last_unstale":"2024-03-29T04:09:22.329391+0000","last_undegraded":"2024-03-29T04:09:22.329391+0000","last_fullsized":"2024-03-29T04:09:22.329391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655284+0000","last_change":"2024-03-29T04:09:16.322983+0000","last_active":"2024-03-29T04:09:22.655284+0000","last_peered":"2024-03-29T04:09:22.655284+0000","last_clean":"2024-03-29T04:09:22.655284+0000","last_became_active":"2024-03-29T04:09:16.322712+0000","last_became_peered":"2024-03-29T04:09:16.322712+0000","last_unstale":"2024-03-29T04:09:22.655284+0000","last_undegraded":"2024-03-29T04:09:22.655284+0000","last_fullsized":"2024-03-29T04:09:22.655284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:15.302835+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:15.302365+0000","last_became_peered":"2024-03-29T04:09:15.302365+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655981+0000","last_change":"2024-03-29T04:09:20.347857+0000","last_active":"2024-03-29T04:09:22.655981+0000","last_peered":"2024-03-29T04:09:22.655981+0000","last_clean":"2024-03-29T04:09:22.655981+0000","last_became_active":"2024-03-29T04:09:20.347499+0000","last_became_peered":"2024-03-29T04:09:20.347499+0000","last_unstale":"2024-03-29T04:09:22.655981+0000","last_undegraded":"2024-03-29T04:09:22.655981+0000","last_fullsized":"2024-03-29T04:09:22.655981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656262+0000","last_change":"2024-03-29T04:09:16.321770+0000","last_active":"2024-03-29T04:09:22.656262+0000","last_peered":"2024-03-29T04:09:22.656262+0000","last_clean":"2024-03-29T04:09:22.656262+0000","last_became_active":"2024-03-29T04:09:16.321511+0000","last_became_peered":"2024-03-29T04:09:16.321511+0000","last_unstale":"2024-03-29T04:09:22.656262+0000","last_undegraded":"2024-03-29T04:09:22.656262+0000","last_fullsized":"2024-03-29T04:09:22.656262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656031+0000","last_change":"2024-03-29T04:09:15.306618+0000","last_active":"2024-03-29T04:09:22.656031+0000","last_peered":"2024-03-29T04:09:22.656031+0000","last_clean":"2024-03-29T04:09:22.656031+0000","last_became_active":"2024-03-29T04:09:15.306337+0000","last_became_peered":"2024-03-29T04:09:15.306337+0000","last_unstale":"2024-03-29T04:09:22.656031+0000","last_undegraded":"2024-03-29T04:09:22.656031+0000","last_fullsized":"2024-03-29T04:09:22.656031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654617+0000","last_change":"2024-03-29T04:09:20.884325+0000","last_active":"2024-03-29T04:09:22.654617+0000","last_peered":"2024-03-29T04:09:22.654617+0000","last_clean":"2024-03-29T04:09:22.654617+0000","last_became_active":"2024-03-29T04:09:20.883509+0000","last_became_peered":"2024-03-29T04:09:20.883509+0000","last_unstale":"2024-03-29T04:09:22.654617+0000","last_undegraded":"2024-03-29T04:09:22.654617+0000","last_fullsized":"2024-03-29T04:09:22.654617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:45:02.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655965+0000","last_change":"2024-03-29T04:09:16.309527+0000","last_active":"2024-03-29T04:09:22.655965+0000","last_peered":"2024-03-29T04:09:22.655965+0000","last_clean":"2024-03-29T04:09:22.655965+0000","last_became_active":"2024-03-29T04:09:16.309219+0000","last_became_peered":"2024-03-29T04:09:16.309219+0000","last_unstale":"2024-03-29T04:09:22.655965+0000","last_undegraded":"2024-03-29T04:09:22.655965+0000","last_fullsized":"2024-03-29T04:09:22.655965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329782+0000","last_change":"2024-03-29T04:09:15.306652+0000","last_active":"2024-03-29T04:09:22.329782+0000","last_peered":"2024-03-29T04:09:22.329782+0000","last_clean":"2024-03-29T04:09:22.329782+0000","last_became_active":"2024-03-29T04:09:15.306354+0000","last_became_peered":"2024-03-29T04:09:15.306354+0000","last_unstale":"2024-03-29T04:09:22.329782+0000","last_undegraded":"2024-03-29T04:09:22.329782+0000","last_fullsized":"2024-03-29T04:09:22.329782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330540+0000","last_change":"2024-03-29T04:09:20.346199+0000","last_active":"2024-03-29T04:09:22.330540+0000","last_peered":"2024-03-29T04:09:22.330540+0000","last_clean":"2024-03-29T04:09:22.330540+0000","last_became_active":"2024-03-29T04:09:20.345826+0000","last_became_peered":"2024-03-29T04:09:20.345826+0000","last_unstale":"2024-03-29T04:09:22.330540+0000","last_undegraded":"2024-03-29T04:09:22.330540+0000","last_fullsized":"2024-03-29T04:09:22.330540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.321785+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.321533+0000","last_became_peered":"2024-03-29T04:09:16.321533+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655385+0000","last_change":"2024-03-29T04:09:15.307664+0000","last_active":"2024-03-29T04:09:22.655385+0000","last_peered":"2024-03-29T04:09:22.655385+0000","last_clean":"2024-03-29T04:09:22.655385+0000","last_became_active":"2024-03-29T04:09:15.307412+0000","last_became_peered":"2024-03-29T04:09:15.307412+0000","last_unstale":"2024-03-29T04:09:22.655385+0000","last_undegraded":"2024-03-29T04:09:22.655385+0000","last_fullsized":"2024-03-29T04:09:22.655385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655742+0000","last_change":"2024-03-29T04:09:20.346476+0000","last_active":"2024-03-29T04:09:22.655742+0000","last_peered":"2024-03-29T04:09:22.655742+0000","last_clean":"2024-03-29T04:09:22.655742+0000","last_became_active":"2024-03-29T04:09:20.346140+0000","last_became_peered":"2024-03-29T04:09:20.346140+0000","last_unstale":"2024-03-29T04:09:22.655742+0000","last_undegraded":"2024-03-29T04:09:22.655742+0000","last_fullsized":"2024-03-29T04:09:22.655742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329068+0000","last_change":"2024-03-29T04:09:16.313068+0000","last_active":"2024-03-29T04:09:22.329068+0000","last_peered":"2024-03-29T04:09:22.329068+0000","last_clean":"2024-03-29T04:09:22.329068+0000","last_became_active":"2024-03-29T04:09:16.312754+0000","last_became_peered":"2024-03-29T04:09:16.312754+0000","last_unstale":"2024-03-29T04:09:22.329068+0000","last_undegraded":"2024-03-29T04:09:22.329068+0000","last_fullsized":"2024-03-29T04:09:22.329068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329238+0000","last_change":"2024-03-29T04:09:15.300287+0000","last_active":"2024-03-29T04:09:22.329238+0000","last_peered":"2024-03-29T04:09:22.329238+0000","last_clean":"2024-03-29T04:09:22.329238+0000","last_became_active":"2024-03-29T04:09:15.299961+0000","last_became_peered":"2024-03-29T04:09:15.299961+0000","last_unstale":"2024-03-29T04:09:22.329238+0000","last_undegraded":"2024-03-29T04:09:22.329238+0000","last_fullsized":"2024-03-29T04:09:22.329238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329627+0000","last_change":"2024-03-29T04:09:20.883520+0000","last_active":"2024-03-29T04:09:22.329627+0000","last_peered":"2024-03-29T04:09:22.329627+0000","last_clean":"2024-03-29T04:09:22.329627+0000","last_became_active":"2024-03-29T04:09:20.883152+0000","last_became_peered":"2024-03-29T04:09:20.883152+0000","last_unstale":"2024-03-29T04:09:22.329627+0000","last_undegraded":"2024-03-29T04:09:22.329627+0000","last_fullsized":"2024-03-29T04:09:22.329627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:25.394492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655644+0000","last_change":"2024-03-29T04:09:16.322576+0000","last_active":"2024-03-29T04:09:22.655644+0000","last_peered":"2024-03-29T04:09:22.655644+0000","last_clean":"2024-03-29T04:09:22.655644+0000","last_became_active":"2024-03-29T04:09:16.322126+0000","last_became_peered":"2024-03-29T04:09:16.322126+0000","last_unstale":"2024-03-29T04:09:22.655644+0000","last_undegraded":"2024-03-29T04:09:22.655644+0000","last_fullsized":"2024-03-29T04:09:22.655644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655221+0000","last_change":"2024-03-29T04:09:15.306114+0000","last_active":"2024-03-29T04:09:22.655221+0000","last_peered":"2024-03-29T04:09:22.655221+0000","last_clean":"2024-03-29T04:09:22.655221+0000","last_became_active":"2024-03-29T04:09:15.305893+0000","last_became_peered":"2024-03-29T04:09:15.305893+0000","last_unstale":"2024-03-29T04:09:22.655221+0000","last_undegraded":"2024-03-29T04:09:22.655221+0000","last_fullsized":"2024-03-29T04:09:22.655221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329246+0000","last_change":"2024-03-29T04:09:20.882277+0000","last_active":"2024-03-29T04:09:22.329246+0000","last_peered":"2024-03-29T04:09:22.329246+0000","last_clean":"2024-03-29T04:09:22.329246+0000","last_became_active":"2024-03-29T04:09:20.881878+0000","last_became_peered":"2024-03-29T04:09:20.881878+0000","last_unstale":"2024-03-29T04:09:22.329246+0000","last_undegraded":"2024-03-29T04:09:22.329246+0000","last_fullsized":"2024-03-29T04:09:22.329246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:06.012938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655785+0000","last_change":"2024-03-29T04:09:16.307882+0000","last_active":"2024-03-29T04:09:22.655785+0000","last_peered":"2024-03-29T04:09:22.655785+0000","last_clean":"2024-03-29T04:09:22.655785+0000","last_became_active":"2024-03-29T04:09:16.307620+0000","last_became_peered":"2024-03-29T04:09:16.307620+0000","last_unstale":"2024-03-29T04:09:22.655785+0000","last_undegraded":"2024-03-29T04:09:22.655785+0000","last_fullsized":"2024-03-29T04:09:22.655785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656164+0000","last_change":"2024-03-29T04:09:15.300985+0000","last_active":"2024-03-29T04:09:22.656164+0000","last_peered":"2024-03-29T04:09:22.656164+0000","last_clean":"2024-03-29T04:09:22.656164+0000","last_became_active":"2024-03-29T04:09:15.300603+0000","last_became_peered":"2024-03-29T04:09:15.300603+0000","last_unstale":"2024-03-29T04:09:22.656164+0000","last_undegraded":"2024-03-29T04:09:22.656164+0000","last_fullsized":"2024-03-29T04:09:22.656164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:12.516429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655363+0000","last_change":"2024-03-29T04:09:20.883953+0000","last_active":"2024-03-29T04:09:22.655363+0000","last_peered":"2024-03-29T04:09:22.655363+0000","last_clean":"2024-03-29T04:09:22.655363+0000","last_became_active":"2024-03-29T04:09:20.883597+0000","last_became_peered":"2024-03-29T04:09:20.883597+0000","last_unstale":"2024-03-29T04:09:22.655363+0000","last_undegraded":"2024-03-29T04:09:22.655363+0000","last_fullsized":"2024-03-29T04:09:22.655363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:27.295430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655864+0000","last_change":"2024-03-29T04:09:16.306905+0000","last_active":"2024-03-29T04:09:22.655864+0000","last_peered":"2024-03-29T04:09:22.655864+0000","last_clean":"2024-03-29T04:09:22.655864+0000","last_became_active":"2024-03-29T04:09:16.306156+0000","last_became_peered":"2024-03-29T04:09:16.306156+0000","last_unstale":"2024-03-29T04:09:22.655864+0000","last_undegraded":"2024-03-29T04:09:22.655864+0000","last_fullsized":"2024-03-29T04:09:22.655864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:25.831236+0000","last_change":"2024-03-29T04:09:15.302359+0000","last_active":"2024-03-29T04:09:25.831236+0000","last_peered":"2024-03-29T04:09:25.831236+0000","last_clean":"2024-03-29T04:09:25.831236+0000","last_became_active":"2024-03-29T04:09:15.302044+0000","last_became_peered":"2024-03-29T04:09:15.302044+0000","last_unstale":"2024-03-29T04:09:25.831236+0000","last_undegraded":"2024-03-29T04:09:25.831236+0000","last_fullsized":"2024-03-29T04:09:25.831236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656202+0000","last_change":"2024-03-29T04:09:20.352101+0000","last_active":"2024-03-29T04:09:22.656202+0000","last_peered":"2024-03-29T04:09:22.656202+0000","last_clean":"2024-03-29T04:09:22.656202+0000","last_became_active":"2024-03-29T04:09:20.351597+0000","last_became_peered":"2024-03-29T04:09:20.351597+0000","last_unstale":"2024-03-29T04:09:22.656202+0000","last_undegraded":"2024-03-29T04:09:22.656202+0000","last_fullsized":"2024-03-29T04:09:22.656202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330245+0000","last_change":"2024-03-29T04:09:16.305800+0000","last_active":"2024-03-29T04:09:22.330245+0000","last_peered":"2024-03-29T04:09:22.330245+0000","last_clean":"2024-03-29T04:09:22.330245+0000","last_became_active":"2024-03-29T04:09:16.305534+0000","last_became_peered":"2024-03-29T04:09:16.305534+0000","last_unstale":"2024-03-29T04:09:22.330245+0000","last_undegraded":"2024-03-29T04:09:22.330245+0000","last_fullsized":"2024-03-29T04:09:22.330245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655674+0000","last_change":"2024-03-29T04:09:15.309193+0000","last_active":"2024-03-29T04:09:22.655674+0000","last_peered":"2024-03-29T04:09:22.655674+0000","last_clean":"2024-03-29T04:09:22.655674+0000","last_became_active":"2024-03-29T04:09:15.308884+0000","last_became_peered":"2024-03-29T04:09:15.308884+0000","last_unstale":"2024-03-29T04:09:22.655674+0000","last_undegraded":"2024-03-29T04:09:22.655674+0000","last_fullsized":"2024-03-29T04:09:22.655674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330135+0000","last_change":"2024-03-29T04:09:20.348029+0000","last_active":"2024-03-29T04:09:22.330135+0000","last_peered":"2024-03-29T04:09:22.330135+0000","last_clean":"2024-03-29T04:09:22.330135+0000","last_became_active":"2024-03-29T04:09:20.347196+0000","last_became_peered":"2024-03-29T04:09:20.347196+0000","last_unstale":"2024-03-29T04:09:22.330135+0000","last_undegraded":"2024-03-29T04:09:22.330135+0000","last_fullsized":"2024-03-29T04:09:22.330135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656194+0000","last_change":"2024-03-29T04:09:16.318939+0000","last_active":"2024-03-29T04:09:22.656194+0000","last_peered":"2024-03-29T04:09:22.656194+0000","last_clean":"2024-03-29T04:09:22.656194+0000","last_became_active":"2024-03-29T04:09:16.318327+0000","last_became_peered":"2024-03-29T04:09:16.318327+0000","last_unstale":"2024-03-29T04:09:22.656194+0000","last_undegraded":"2024-03-29T04:09:22.656194+0000","last_fullsized":"2024-03-29T04:09:22.656194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654888+0000","last_change":"2024-03-29T04:09:15.300878+0000","last_active":"2024-03-29T04:09:22.654888+0000","last_peered":"2024-03-29T04:09:22.654888+0000","last_clean":"2024-03-29T04:09:22.654888+0000","last_became_active":"2024-03-29T04:09:15.300567+0000","last_became_peered":"2024-03-29T04:09:15.300567+0000","last_unstale":"2024-03-29T04:09:22.654888+0000","last_undegraded":"2024-03-29T04:09:22.654888+0000","last_fullsized":"2024-03-29T04:09:22.654888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328959+0000","last_change":"2024-03-29T04:09:20.343510+0000","last_active":"2024-03-29T04:09:22.328959+0000","last_peered":"2024-03-29T04:09:22.328959+0000","last_clean":"2024-03-29T04:09:22.328959+0000","last_became_active":"2024-03-29T04:09:20.343070+0000","last_became_peered":"2024-03-29T04:09:20.343070+0000","last_unstale":"2024-03-29T04:09:22.328959+0000","last_undegraded":"2024-03-29T04:09:22.328959+0000","last_fullsized":"2024-03-29T04:09:22.328959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:10.010570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654868+0000","last_change":"2024-03-29T04:09:16.314479+0000","last_active":"2024-03-29T04:09:22.654868+0000","last_peered":"2024-03-29T04:09:22.654868+0000","last_clean":"2024-03-29T04:09:22.654868+0000","last_became_active":"2024-03-29T04:09:16.313990+0000","last_became_peered":"2024-03-29T04:09:16.313990+0000","last_unstale":"2024-03-29T04:09:22.654868+0000","last_undegraded":"2024-03-29T04:09:22.654868+0000","last_fullsized":"2024-03-29T04:09:22.654868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329948+0000","last_change":"2024-03-29T04:09:15.304665+0000","last_active":"2024-03-29T04:09:22.329948+0000","last_peered":"2024-03-29T04:09:22.329948+0000","last_clean":"2024-03-29T04:09:22.329948+0000","last_became_active":"2024-03-29T04:09:15.304381+0000","last_became_peered":"2024-03-29T04:09:15.304381+0000","last_unstale":"2024-03-29T04:09:22.329948+0000","last_undegraded":"2024-03-29T04:09:22.329948+0000","last_fullsized":"2024-03-29T04:09:22.329948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329035+0000","last_change":"2024-03-29T04:09:20.884477+0000","last_active":"2024-03-29T04:09:22.329035+0000","last_peered":"2024-03-29T04:09:22.329035+0000","last_clean":"2024-03-29T04:09:22.329035+0000","last_became_active":"2024-03-29T04:09:20.884123+0000","last_became_peered":"2024-03-29T04:09:20.884123+0000","last_unstale":"2024-03-29T04:09:22.329035+0000","last_undegraded":"2024-03-29T04:09:22.329035+0000","last_fullsized":"2024-03-29T04:09:22.329035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:11.194987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655599+0000","last_change":"2024-03-29T04:09:20.883017+0000","last_active":"2024-03-29T04:09:22.655599+0000","last_peered":"2024-03-29T04:09:22.655599+0000","last_clean":"2024-03-29T04:09:22.655599+0000","last_became_active":"2024-03-29T04:09:20.882655+0000","last_became_peered":"2024-03-29T04:09:20.882655+0000","last_unstale":"2024-03-29T04:09:22.655599+0000","last_undegraded":"2024-03-29T04:09:22.655599+0000","last_fullsized":"2024-03-29T04:09:22.655599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655207+0000","last_change":"2024-03-29T04:09:16.313418+0000","last_active":"2024-03-29T04:09:22.655207+0000","last_peered":"2024-03-29T04:09:22.655207+0000","last_clean":"2024-03-29T04:09:22.655207+0000","last_became_active":"2024-03-29T04:09:16.312923+0000","last_became_peered":"2024-03-29T04:09:16.312923+0000","last_unstale":"2024-03-29T04:09:22.655207+0000","last_undegraded":"2024-03-29T04:09:22.655207+0000","last_fullsized":"2024-03-29T04:09:22.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654732+0000","last_change":"2024-03-29T04:09:15.304449+0000","last_active":"2024-03-29T04:09:22.654732+0000","last_peered":"2024-03-29T04:09:22.654732+0000","last_clean":"2024-03-29T04:09:22.654732+0000","last_became_active":"2024-03-29T04:09:15.304164+0000","last_became_peered":"2024-03-29T04:09:15.304164+0000","last_unstale":"2024-03-29T04:09:22.654732+0000","last_undegraded":"2024-03-29T04:09:22.654732+0000","last_fullsized":"2024-03-29T04:09:22.654732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329921+0000","last_change":"2024-03-29T04:09:20.344669+0000","last_active":"2024-03-29T04:09:22.329921+0000","last_peered":"2024-03-29T04:09:22.329921+0000","last_clean":"2024-03-29T04:09:22.329921+0000","last_became_active":"2024-03-29T04:09:20.344359+0000","last_became_peered":"2024-03-29T04:09:20.344359+0000","last_unstale":"2024-03-29T04:09:22.329921+0000","last_undegraded":"2024-03-29T04:09:22.329921+0000","last_fullsized":"2024-03-29T04:09:22.329921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328865+0000","last_change":"2024-03-29T04:09:16.308421+0000","last_active":"2024-03-29T04:09:22.328865+0000","last_peered":"2024-03-29T04:09:22.328865+0000","last_clean":"2024-03-29T04:09:22.328865+0000","last_became_active":"2024-03-29T04:09:16.308142+0000","last_became_peered":"2024-03-29T04:09:16.308142+0000","last_unstale":"2024-03-29T04:09:22.328865+0000","last_undegraded":"2024-03-29T04:09:22.328865+0000","last_fullsized":"2024-03-29T04:09:22.328865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329237+0000","last_change":"2024-03-29T04:09:15.302691+0000","last_active":"2024-03-29T04:09:22.329237+0000","last_peered":"2024-03-29T04:09:22.329237+0000","last_clean":"2024-03-29T04:09:22.329237+0000","last_became_active":"2024-03-29T04:09:15.302305+0000","last_became_peered":"2024-03-29T04:09:15.302305+0000","last_unstale":"2024-03-29T04:09:22.329237+0000","last_undegraded":"2024-03-29T04:09:22.329237+0000","last_fullsized":"2024-03-29T04:09:22.329237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329606+0000","last_change":"2024-03-29T04:09:15.299655+0000","last_active":"2024-03-29T04:09:22.329606+0000","last_peered":"2024-03-29T04:09:22.329606+0000","last_clean":"2024-03-29T04:09:22.329606+0000","last_became_active":"2024-03-29T04:09:15.299361+0000","last_became_peered":"2024-03-29T04:09:15.299361+0000","last_unstale":"2024-03-29T04:09:22.329606+0000","last_undegraded":"2024-03-29T04:09:22.329606+0000","last_fullsized":"2024-03-29T04:09:22.329606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656124+0000","last_change":"2024-03-29T04:09:16.322096+0000","last_active":"2024-03-29T04:09:22.656124+0000","last_peered":"2024-03-29T04:09:22.656124+0000","last_clean":"2024-03-29T04:09:22.656124+0000","last_became_active":"2024-03-29T04:09:16.321833+0000","last_became_peered":"2024-03-29T04:09:16.321833+0000","last_unstale":"2024-03-29T04:09:22.656124+0000","last_undegraded":"2024-03-29T04:09:22.656124+0000","last_fullsized":"2024-03-29T04:09:22.656124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656607+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.656607+0000","last_peered":"2024-03-29T04:09:22.656607+0000","last_clean":"2024-03-29T04:09:22.656607+0000","last_became_active":"2024-03-29T04:09:20.347088+0000","last_became_peered":"2024-03-29T04:09:20.347088+0000","last_unstale":"2024-03-29T04:09:22.656607+0000","last_undegraded":"2024-03-29T04:09:22.656607+0000","last_fullsized":"2024-03-29T04:09:22.656607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:45.089535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329185+0000","last_change":"2024-03-29T04:09:15.301823+0000","last_active":"2024-03-29T04:09:22.329185+0000","last_peered":"2024-03-29T04:09:22.329185+0000","last_clean":"2024-03-29T04:09:22.329185+0000","last_became_active":"2024-03-29T04:09:15.301182+0000","last_became_peered":"2024-03-29T04:09:15.301182+0000","last_unstale":"2024-03-29T04:09:22.329185+0000","last_undegraded":"2024-03-29T04:09:22.329185+0000","last_fullsized":"2024-03-29T04:09:22.329185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329886+0000","last_change":"2024-03-29T04:09:16.311797+0000","last_active":"2024-03-29T04:09:22.329886+0000","last_peered":"2024-03-29T04:09:22.329886+0000","last_clean":"2024-03-29T04:09:22.329886+0000","last_became_active":"2024-03-29T04:09:16.311544+0000","last_became_peered":"2024-03-29T04:09:16.311544+0000","last_unstale":"2024-03-29T04:09:22.329886+0000","last_undegraded":"2024-03-29T04:09:22.329886+0000","last_fullsized":"2024-03-29T04:09:22.329886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:41.184352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329592+0000","last_change":"2024-03-29T04:09:20.344647+0000","last_active":"2024-03-29T04:09:22.329592+0000","last_peered":"2024-03-29T04:09:22.329592+0000","last_clean":"2024-03-29T04:09:22.329592+0000","last_became_active":"2024-03-29T04:09:20.344278+0000","last_became_peered":"2024-03-29T04:09:20.344278+0000","last_unstale":"2024-03-29T04:09:22.329592+0000","last_undegraded":"2024-03-29T04:09:22.329592+0000","last_fullsized":"2024-03-29T04:09:22.329592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:15.627313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328975+0000","last_change":"2024-03-29T04:09:16.311872+0000","last_active":"2024-03-29T04:09:22.328975+0000","last_peered":"2024-03-29T04:09:22.328975+0000","last_clean":"2024-03-29T04:09:22.328975+0000","last_became_active":"2024-03-29T04:09:16.311576+0000","last_became_peered":"2024-03-29T04:09:16.311576+0000","last_unstale":"2024-03-29T04:09:22.328975+0000","last_undegraded":"2024-03-29T04:09:22.328975+0000","last_fullsized":"2024-03-29T04:09:22.328975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656255+0000","last_change":"2024-03-29T04:09:16.315268+0000","last_active":"2024-03-29T04:09:22.656255+0000","last_peered":"2024-03-29T04:09:22.656255+0000","last_clean":"2024-03-29T04:09:22.656255+0000","last_became_active":"2024-03-29T04:09:16.315002+0000","last_became_peered":"2024-03-29T04:09:16.315002+0000","last_unstale":"2024-03-29T04:09:22.656255+0000","last_undegraded":"2024-03-29T04:09:22.656255+0000","last_fullsized":"2024-03-29T04:09:22.656255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.323782+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.323504+0000","last_became_peered":"2024-03-29T04:09:16.323504+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656037+0000","last_change":"2024-03-29T04:09:16.316947+0000","last_active":"2024-03-29T04:09:22.656037+0000","last_peered":"2024-03-29T04:09:22.656037+0000","last_clean":"2024-03-29T04:09:22.656037+0000","last_became_active":"2024-03-29T04:09:16.316689+0000","last_became_peered":"2024-03-29T04:09:16.316689+0000","last_unstale":"2024-03-29T04:09:22.656037+0000","last_undegraded":"2024-03-29T04:09:22.656037+0000","last_fullsized":"2024-03-29T04:09:22.656037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329288+0000","last_change":"2024-03-29T04:09:16.319962+0000","last_active":"2024-03-29T04:09:22.329288+0000","last_peered":"2024-03-29T04:09:22.329288+0000","last_clean":"2024-03-29T04:09:22.329288+0000","last_became_active":"2024-03-29T04:09:16.319316+0000","last_became_peered":"2024-03-29T04:09:16.319316+0000","last_unstale":"2024-03-29T04:09:22.329288+0000","last_undegraded":"2024-03-29T04:09:22.329288+0000","last_fullsized":"2024-03-29T04:09:22.329288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655509+0000","last_change":"2024-03-29T04:09:16.312227+0000","last_active":"2024-03-29T04:09:22.655509+0000","last_peered":"2024-03-29T04:09:22.655509+0000","last_clean":"2024-03-29T04:09:22.655509+0000","last_became_active":"2024-03-29T04:09:16.311944+0000","last_became_peered":"2024-03-29T04:09:16.311944+0000","last_unstale":"2024-03-29T04:09:22.655509+0000","last_undegraded":"2024-03-29T04:09:22.655509+0000","last_fullsized":"2024-03-29T04:09:22.655509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330212+0000","last_change":"2024-03-29T04:09:16.306234+0000","last_active":"2024-03-29T04:09:22.330212+0000","last_peered":"2024-03-29T04:09:22.330212+0000","last_clean":"2024-03-29T04:09:22.330212+0000","last_became_active":"2024-03-29T04:09:16.305931+0000","last_became_peered":"2024-03-29T04:09:16.305931+0000","last_unstale":"2024-03-29T04:09:22.330212+0000","last_undegraded":"2024-03-29T04:09:22.330212+0000","last_fullsized":"2024-03-29T04:09:22.330212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329471+0000","last_change":"2024-03-29T04:09:16.310048+0000","last_active":"2024-03-29T04:09:22.329471+0000","last_peered":"2024-03-29T04:09:22.329471+0000","last_clean":"2024-03-29T04:09:22.329471+0000","last_became_active":"2024-03-29T04:09:16.309626+0000","last_became_peered":"2024-03-29T04:09:16.309626+0000","last_unstale":"2024-03-29T04:09:22.329471+0000","last_undegraded":"2024-03-29T04:09:22.329471+0000","last_fullsized":"2024-03-29T04:09:22.329471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654709+0000","last_change":"2024-03-29T04:09:16.324820+0000","last_active":"2024-03-29T04:09:22.654709+0000","last_peered":"2024-03-29T04:09:22.654709+0000","last_clean":"2024-03-29T04:09:22.654709+0000","last_became_active":"2024-03-29T04:09:16.324516+0000","last_became_peered":"2024-03-29T04:09:16.324516+0000","last_unstale":"2024-03-29T04:09:22.654709+0000","last_undegraded":"2024-03-29T04:09:22.654709+0000","last_fullsized":"2024-03-29T04:09:22.654709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329113+0000","last_change":"2024-03-29T04:09:16.318107+0000","last_active":"2024-03-29T04:09:22.329113+0000","last_peered":"2024-03-29T04:09:22.329113+0000","last_clean":"2024-03-29T04:09:22.329113+0000","last_became_active":"2024-03-29T04:09:16.317633+0000","last_became_peered":"2024-03-29T04:09:16.317633+0000","last_unstale":"2024-03-29T04:09:22.329113+0000","last_undegraded":"2024-03-29T04:09:22.329113+0000","last_fullsized":"2024-03-29T04:09:22.329113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654968+0000","last_change":"2024-03-29T04:09:16.317756+0000","last_active":"2024-03-29T04:09:22.654968+0000","last_peered":"2024-03-29T04:09:22.654968+0000","last_clean":"2024-03-29T04:09:22.654968+0000","last_became_active":"2024-03-29T04:09:16.317411+0000","last_became_peered":"2024-03-29T04:09:16.317411+0000","last_unstale":"2024-03-29T04:09:22.654968+0000","last_undegraded":"2024-03-29T04:09:22.654968+0000","last_fullsized":"2024-03-29T04:09:22.654968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655375+0000","last_change":"2024-03-29T04:09:16.314033+0000","last_active":"2024-03-29T04:09:22.655375+0000","last_peered":"2024-03-29T04:09:22.655375+0000","last_clean":"2024-03-29T04:09:22.655375+0000","last_became_active":"2024-03-29T04:09:16.313440+0000","last_became_peered":"2024-03-29T04:09:16.313440+0000","last_unstale":"2024-03-29T04:09:22.655375+0000","last_undegraded":"2024-03-29T04:09:22.655375+0000","last_fullsized":"2024-03-29T04:09:22.655375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329544+0000","last_change":"2024-03-29T04:09:16.318843+0000","last_active":"2024-03-29T04:09:22.329544+0000","last_peered":"2024-03-29T04:09:22.329544+0000","last_clean":"2024-03-29T04:09:22.329544+0000","last_became_active":"2024-03-29T04:09:16.318563+0000","last_became_peered":"2024-03-29T04:09:16.318563+0000","last_unstale":"2024-03-29T04:09:22.329544+0000","last_undegraded":"2024-03-29T04:09:22.329544+0000","last_fullsized":"2024-03-29T04:09:22.329544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656080+0000","last_change":"2024-03-29T04:09:16.313733+0000","last_active":"2024-03-29T04:09:22.656080+0000","last_peered":"2024-03-29T04:09:22.656080+0000","last_clean":"2024-03-29T04:09:22.656080+0000","last_became_active":"2024-03-29T04:09:16.313193+0000","last_became_peered":"2024-03-29T04:09:16.313193+0000","last_unstale":"2024-03-29T04:09:22.656080+0000","last_undegraded":"2024-03-29T04:09:22.656080+0000","last_fullsized":"2024-03-29T04:09:22.656080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:59.588666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655943+0000","last_change":"2024-03-29T04:09:16.300679+0000","last_active":"2024-03-29T04:09:22.655943+0000","last_peered":"2024-03-29T04:09:22.655943+0000","last_clean":"2024-03-29T04:09:22.655943+0000","last_became_active":"2024-03-29T04:09:16.300399+0000","last_became_peered":"2024-03-29T04:09:16.300399+0000","last_unstale":"2024-03-29T04:09:22.655943+0000","last_undegraded":"2024-03-29T04:09:22.655943+0000","last_fullsized":"2024-03-29T04:09:22.655943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:10.179988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655462+0000","last_change":"2024-03-29T04:09:16.314857+0000","last_active":"2024-03-29T04:09:22.655462+0000","last_peered":"2024-03-29T04:09:22.655462+0000","last_clean":"2024-03-29T04:09:22.655462+0000","last_became_active":"2024-03-29T04:09:16.314478+0000","last_became_peered":"2024-03-29T04:09:16.314478+0000","last_unstale":"2024-03-29T04:09:22.655462+0000","last_undegraded":"2024-03-29T04:09:22.655462+0000","last_fullsized":"2024-03-29T04:09:22.655462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654626+0000","last_change":"2024-03-29T04:09:16.314396+0000","last_active":"2024-03-29T04:09:22.654626+0000","last_peered":"2024-03-29T04:09:22.654626+0000","last_clean":"2024-03-29T04:09:22.654626+0000","last_became_active":"2024-03-29T04:09:16.313977+0000","last_became_peered":"2024-03-29T04:09:16.313977+0000","last_unstale":"2024-03-29T04:09:22.654626+0000","last_undegraded":"2024-03-29T04:09:22.654626+0000","last_fullsized":"2024-03-29T04:09:22.654626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655312+0000","last_change":"2024-03-29T04:09:16.320206+0000","last_active":"2024-03-29T04:09:22.655312+0000","last_peered":"2024-03-29T04:09:22.655312+0000","last_clean":"2024-03-29T04:09:22.655312+0000","last_became_active":"2024-03-29T04:09:16.319909+0000","last_became_peered":"2024-03-29T04:09:16.319909+0000","last_unstale":"2024-03-29T04:09:22.655312+0000","last_undegraded":"2024-03-29T04:09:22.655312+0000","last_fullsized":"2024-03-29T04:09:22.655312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:16.308762+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:16.308446+0000","last_became_peered":"2024-03-29T04:09:16.308446+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:27.406089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328992+0000","last_change":"2024-03-29T04:09:16.306843+0000","last_active":"2024-03-29T04:09:22.328992+0000","last_peered":"2024-03-29T04:09:22.328992+0000","last_clean":"2024-03-29T04:09:22.328992+0000","last_became_active":"2024-03-29T04:09:16.306402+0000","last_became_peered":"2024-03-29T04:09:16.306402+0000","last_unstale":"2024-03-29T04:09:22.328992+0000","last_undegraded":"2024-03-29T04:09:22.328992+0000","last_fullsized":"2024-03-29T04:09:22.328992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330172+0000","last_change":"2024-03-29T04:09:16.310846+0000","last_active":"2024-03-29T04:09:22.330172+0000","last_peered":"2024-03-29T04:09:22.330172+0000","last_clean":"2024-03-29T04:09:22.330172+0000","last_became_active":"2024-03-29T04:09:16.309820+0000","last_became_peered":"2024-03-29T04:09:16.309820+0000","last_unstale":"2024-03-29T04:09:22.330172+0000","last_undegraded":"2024-03-29T04:09:22.330172+0000","last_fullsized":"2024-03-29T04:09:22.330172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:18.601851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329788+0000","last_change":"2024-03-29T04:09:16.316170+0000","last_active":"2024-03-29T04:09:22.329788+0000","last_peered":"2024-03-29T04:09:22.329788+0000","last_clean":"2024-03-29T04:09:22.329788+0000","last_became_active":"2024-03-29T04:09:16.315896+0000","last_became_peered":"2024-03-29T04:09:16.315896+0000","last_unstale":"2024-03-29T04:09:22.329788+0000","last_undegraded":"2024-03-29T04:09:22.329788+0000","last_fullsized":"2024-03-29T04:09:22.329788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330150+0000","last_change":"2024-03-29T04:09:16.316842+0000","last_active":"2024-03-29T04:09:22.330150+0000","last_peered":"2024-03-29T04:09:22.330150+0000","last_clean":"2024-03-29T04:09:22.330150+0000","last_became_active":"2024-03-29T04:09:16.316573+0000","last_became_peered":"2024-03-29T04:09:16.316573+0000","last_unstale":"2024-03-29T04:09:22.330150+0000","last_undegraded":"2024-03-29T04:09:22.330150+0000","last_fullsized":"2024-03-29T04:09:22.330150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656222+0000","last_change":"2024-03-29T04:09:16.316209+0000","last_active":"2024-03-29T04:09:22.656222+0000","last_peered":"2024-03-29T04:09:22.656222+0000","last_clean":"2024-03-29T04:09:22.656222+0000","last_became_active":"2024-03-29T04:09:16.315942+0000","last_became_peered":"2024-03-29T04:09:16.315942+0000","last_unstale":"2024-03-29T04:09:22.656222+0000","last_undegraded":"2024-03-29T04:09:22.656222+0000","last_fullsized":"2024-03-29T04:09:22.656222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328767+0000","last_change":"2024-03-29T04:09:16.316731+0000","last_active":"2024-03-29T04:09:22.328767+0000","last_peered":"2024-03-29T04:09:22.328767+0000","last_clean":"2024-03-29T04:09:22.328767+0000","last_became_active":"2024-03-29T04:09:16.316418+0000","last_became_peered":"2024-03-29T04:09:16.316418+0000","last_unstale":"2024-03-29T04:09:22.328767+0000","last_undegraded":"2024-03-29T04:09:22.328767+0000","last_fullsized":"2024-03-29T04:09:22.328767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.312822+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.312495+0000","last_became_peered":"2024-03-29T04:09:16.312495+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654882+0000","last_change":"2024-03-29T04:09:16.316248+0000","last_active":"2024-03-29T04:09:22.654882+0000","last_peered":"2024-03-29T04:09:22.654882+0000","last_clean":"2024-03-29T04:09:22.654882+0000","last_became_active":"2024-03-29T04:09:16.315901+0000","last_became_peered":"2024-03-29T04:09:16.315901+0000","last_unstale":"2024-03-29T04:09:22.654882+0000","last_undegraded":"2024-03-29T04:09:22.654882+0000","last_fullsized":"2024-03-29T04:09:22.654882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655574+0000","last_change":"2024-03-29T04:09:16.307754+0000","last_active":"2024-03-29T04:09:22.655574+0000","last_peered":"2024-03-29T04:09:22.655574+0000","last_clean":"2024-03-29T04:09:22.655574+0000","last_became_active":"2024-03-29T04:09:16.307322+0000","last_became_peered":"2024-03-29T04:09:16.307322+0000","last_unstale":"2024-03-29T04:09:22.655574+0000","last_undegraded":"2024-03-29T04:09:22.655574+0000","last_fullsized":"2024-03-29T04:09:22.655574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.323875+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.323578+0000","last_became_peered":"2024-03-29T04:09:16.323578+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655497+0000","last_change":"2024-03-29T04:09:16.311321+0000","last_active":"2024-03-29T04:09:22.655497+0000","last_peered":"2024-03-29T04:09:22.655497+0000","last_clean":"2024-03-29T04:09:22.655497+0000","last_became_active":"2024-03-29T04:09:16.310990+0000","last_became_peered":"2024-03-29T04:09:16.310990+0000","last_unstale":"2024-03-29T04:09:22.655497+0000","last_undegraded":"2024-03-29T04:09:22.655497+0000","last_fullsized":"2024-03-29T04:09:22.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656870+0000","last_change":"2024-03-29T04:09:16.313513+0000","last_active":"2024-03-29T04:09:22.656870+0000","last_peered":"2024-03-29T04:09:22.656870+0000","last_clean":"2024-03-29T04:09:22.656870+0000","last_became_active":"2024-03-29T04:09:16.312390+0000","last_became_peered":"2024-03-29T04:09:16.312390+0000","last_unstale":"2024-03-29T04:09:22.656870+0000","last_undegraded":"2024-03-29T04:09:22.656870+0000","last_fullsized":"2024-03-29T04:09:22.656870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656162+0000","last_change":"2024-03-29T04:09:16.303994+0000","last_active":"2024-03-29T04:09:22.656162+0000","last_peered":"2024-03-29T04:09:22.656162+0000","last_clean":"2024-03-29T04:09:22.656162+0000","last_became_active":"2024-03-29T04:09:16.303670+0000","last_became_peered":"2024-03-29T04:09:16.303670+0000","last_unstale":"2024-03-29T04:09:22.656162+0000","last_undegraded":"2024-03-29T04:09:22.656162+0000","last_fullsized":"2024-03-29T04:09:22.656162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:37.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:09:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.814+0000 7f532290e640 1 Processor -- start 2024-03-29T04:09:37.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.814+0000 7f532290e640 1 -- start start 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f532290e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f531c0ce6d0 0x7f531c0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f532290e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c15ff80 con 0x7f531c0ceed0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f532290e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c160260 con 0x7f531c0b3cd0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f532290e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f531c0ccfb0 con 0x7f531c0ce6d0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531bfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f531c0ceed0 0x7f531c0cc8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34678/0 (socket says 172.21.15.181:34678) 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531bfff640 1 -- 172.21.15.181:0/2901473231 learned_addr learned my addr 172.21.15.181:0/2901473231 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531b7fe640 1 --2- 172.21.15.181:0/2901473231 >> v2:172.21.15.187:3300/0 conn(0x7f531c0ce6d0 0x7f531c0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639408293 0 0) 0x7f531c15ff80 con 0x7f531c0ceed0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5304003520 con 0x7f531c0ceed0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183383142 0 0) 0x7f531c160260 con 0x7f531c0b3cd0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f531c15ff80 con 0x7f531c0b3cd0 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531b7fe640 1 -- 172.21.15.181:0/2901473231 >> v1:172.21.15.187:6789/0 conn(0x7f531c0b3cd0 legacy=0x7f531c0b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531b7fe640 1 -- 172.21.15.181:0/2901473231 >> v1:172.21.15.181:6789/0 conn(0x7f531c0ceed0 legacy=0x7f531c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.815+0000 7f531b7fe640 1 -- 172.21.15.181:0/2901473231 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c160540 con 0x7f531c0ce6d0 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.816+0000 7f531b7fe640 1 --2- 172.21.15.181:0/2901473231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ce6d0 0x7f531c0d16c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53100097a0 tx=0x7f531002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77400c2c3ec00cee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.816+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310034070 con 0x7f531c0ce6d0 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.816+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5310030610 con 0x7f531c0ce6d0 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.816+0000 7f531affd640 1 -- 172.21.15.181:0/2901473231 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53100309e0 con 0x7f531c0ce6d0 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 -- 172.21.15.181:0/2901473231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ce6d0 msgr2=0x7f531c0d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 --2- 172.21.15.181:0/2901473231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ce6d0 0x7f531c0d16c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53100097a0 tx=0x7f531002b960 comp rx=0 tx=0).stop 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 -- 172.21.15.181:0/2901473231 shutdown_connections 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 --2- 172.21.15.181:0/2901473231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ce6d0 0x7f531c0d16c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 -- 172.21.15.181:0/2901473231 >> 172.21.15.181:0/2901473231 conn(0x7f531c06f820 msgr2=0x7f531c06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 -- 172.21.15.181:0/2901473231 shutdown_connections 2024-03-29T04:09:37.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.817+0000 7f532290e640 1 -- 172.21.15.181:0/2901473231 wait complete. 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.818+0000 7f532290e640 1 Processor -- start 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.818+0000 7f532290e640 1 -- start start 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c0b3cd0 0x7f531c1485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ceed0 0x7f531c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0cf2b0 con 0x7f531c0ce6d0 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f531c160b00 con 0x7f531c0ceed0 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f532290e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f531c0cf430 con 0x7f531c0b3cd0 2024-03-29T04:09:37.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32770/0 (socket says 172.21.15.181:32770) 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 -- 172.21.15.181:0/2611209637 learned_addr learned my addr 172.21.15.181:0/2611209637 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531bfff640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c0b3cd0 0x7f531c1485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f5320eab640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ceed0 0x7f531c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 -- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c0b3cd0 msgr2=0x7f531c1485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c0b3cd0 0x7f531c1485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 -- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ceed0 msgr2=0x7f531c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ceed0 0x7f531c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 -- 172.21.15.181:0/2611209637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c0b5a00 con 0x7f531c0ce6d0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.819+0000 7f531b7fe640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f531002bf70 tx=0x7f53100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.820+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310034040 con 0x7f531c0ce6d0 2024-03-29T04:09:37.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.820+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5310030410 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.820+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310006e10 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.820+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c0b5ce0 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.820+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c0b62a0 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.821+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c075a10 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.821+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5310037070 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.821+0000 7f5318ff9640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 0x7f530806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.822+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f53100ad700 con 0x7f531c0ce6d0 2024-03-29T04:09:37.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.822+0000 7f531bfff640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 0x7f530806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:37.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.822+0000 7f531bfff640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 0x7f530806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f530c000f60 tx=0x7f530c007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:37.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.826+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f531007b420 con 0x7f531c0ce6d0 2024-03-29T04:09:37.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:37.999+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f531c0719d0 con 0x7f530806c530 2024-03-29T04:09:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.006+0000 7f5318ff9640 1 -- 172.21.15.181:0/2611209637 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f531c0719d0 con 0x7f530806c530 2024-03-29T04:09:37.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:37.991 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 msgr2=0x7f530806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 0x7f530806e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f530c000f60 tx=0x7f530c007d70 comp rx=0 tx=0).stop 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 msgr2=0x7f531c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f531002bf70 tx=0x7f53100046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 shutdown_connections 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c0ceed0 0x7f531c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0ce6d0 0x7f531c0b7830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f530806c530 0x7f530806e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 --2- 172.21.15.181:0/2611209637 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c0b3cd0 0x7f531c1485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:37.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 >> 172.21.15.181:0/2611209637 conn(0x7f531c06f820 msgr2=0x7f531c0b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:37.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 shutdown_connections 2024-03-29T04:09:37.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.010+0000 7f532290e640 1 -- 172.21.15.181:0/2611209637 wait complete. 2024-03-29T04:09:38.000 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-03-29T04:09:37.578328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656335+0000","last_change":"2024-03-29T04:09:15.308724+0000","last_active":"2024-03-29T04:09:22.656335+0000","last_peered":"2024-03-29T04:09:22.656335+0000","last_clean":"2024-03-29T04:09:22.656335+0000","last_became_active":"2024-03-29T04:09:15.308445+0000","last_became_peered":"2024-03-29T04:09:15.308445+0000","last_unstale":"2024-03-29T04:09:22.656335+0000","last_undegraded":"2024-03-29T04:09:22.656335+0000","last_fullsized":"2024-03-29T04:09:22.656335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329448+0000","last_change":"2024-03-29T04:09:16.307371+0000","last_active":"2024-03-29T04:09:22.329448+0000","last_peered":"2024-03-29T04:09:22.329448+0000","last_clean":"2024-03-29T04:09:22.329448+0000","last_became_active":"2024-03-29T04:09:16.307077+0000","last_became_peered":"2024-03-29T04:09:16.307077+0000","last_unstale":"2024-03-29T04:09:22.329448+0000","last_undegraded":"2024-03-29T04:09:22.329448+0000","last_fullsized":"2024-03-29T04:09:22.329448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328852+0000","last_change":"2024-03-29T04:09:20.883785+0000","last_active":"2024-03-29T04:09:22.328852+0000","last_peered":"2024-03-29T04:09:22.328852+0000","last_clean":"2024-03-29T04:09:22.328852+0000","last_became_active":"2024-03-29T04:09:20.883360+0000","last_became_peered":"2024-03-29T04:09:20.883360+0000","last_unstale":"2024-03-29T04:09:22.328852+0000","last_undegraded":"2024-03-29T04:09:22.328852+0000","last_fullsized":"2024-03-29T04:09:22.328852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:46.908030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329590+0000","last_change":"2024-03-29T04:09:15.301577+0000","last_active":"2024-03-29T04:09:22.329590+0000","last_peered":"2024-03-29T04:09:22.329590+0000","last_clean":"2024-03-29T04:09:22.329590+0000","last_became_active":"2024-03-29T04:09:15.301247+0000","last_became_peered":"2024-03-29T04:09:15.301247+0000","last_unstale":"2024-03-29T04:09:22.329590+0000","last_undegraded":"2024-03-29T04:09:22.329590+0000","last_fullsized":"2024-03-29T04:09:22.329590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656042+0000","last_change":"2024-03-29T04:09:16.319168+0000","last_active":"2024-03-29T04:09:22.656042+0000","last_peered":"2024-03-29T04:09:22.656042+0000","last_clean":"2024-03-29T04:09:22.656042+0000","last_became_active":"2024-03-29T04:09:16.318673+0000","last_became_peered":"2024-03-29T04:09:16.318673+0000","last_unstale":"2024-03-29T04:09:22.656042+0000","last_undegraded":"2024-03-29T04:09:22.656042+0000","last_fullsized":"2024-03-29T04:09:22.656042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:41.049934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329629+0000","last_change":"2024-03-29T04:09:20.348184+0000","last_active":"2024-03-29T04:09:22.329629+0000","last_peered":"2024-03-29T04:09:22.329629+0000","last_clean":"2024-03-29T04:09:22.329629+0000","last_became_active":"2024-03-29T04:09:20.347504+0000","last_became_peered":"2024-03-29T04:09:20.347504+0000","last_unstale":"2024-03-29T04:09:22.329629+0000","last_undegraded":"2024-03-29T04:09:22.329629+0000","last_fullsized":"2024-03-29T04:09:22.329629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656067+0000","last_change":"2024-03-29T04:09:15.308597+0000","last_active":"2024-03-29T04:09:22.656067+0000","last_peered":"2024-03-29T04:09:22.656067+0000","last_clean":"2024-03-29T04:09:22.656067+0000","last_became_active":"2024-03-29T04:09:15.308020+0000","last_became_peered":"2024-03-29T04:09:15.308020+0000","last_unstale":"2024-03-29T04:09:22.656067+0000","last_undegraded":"2024-03-29T04:09:22.656067+0000","last_fullsized":"2024-03-29T04:09:22.656067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:40.999639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655201+0000","last_change":"2024-03-29T04:09:16.321774+0000","last_active":"2024-03-29T04:09:22.655201+0000","last_peered":"2024-03-29T04:09:22.655201+0000","last_clean":"2024-03-29T04:09:22.655201+0000","last_became_active":"2024-03-29T04:09:16.321464+0000","last_became_peered":"2024-03-29T04:09:16.321464+0000","last_unstale":"2024-03-29T04:09:22.655201+0000","last_undegraded":"2024-03-29T04:09:22.655201+0000","last_fullsized":"2024-03-29T04:09:22.655201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330320+0000","last_change":"2024-03-29T04:09:20.346472+0000","last_active":"2024-03-29T04:09:22.330320+0000","last_peered":"2024-03-29T04:09:22.330320+0000","last_clean":"2024-03-29T04:09:22.330320+0000","last_became_active":"2024-03-29T04:09:20.346130+0000","last_became_peered":"2024-03-29T04:09:20.346130+0000","last_unstale":"2024-03-29T04:09:22.330320+0000","last_undegraded":"2024-03-29T04:09:22.330320+0000","last_fullsized":"2024-03-29T04:09:22.330320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655700+0000","last_change":"2024-03-29T04:09:15.308906+0000","last_active":"2024-03-29T04:09:22.655700+0000","last_peered":"2024-03-29T04:09:22.655700+0000","last_clean":"2024-03-29T04:09:22.655700+0000","last_became_active":"2024-03-29T04:09:15.308632+0000","last_became_peered":"2024-03-29T04:09:15.308632+0000","last_unstale":"2024-03-29T04:09:22.655700+0000","last_undegraded":"2024-03-29T04:09:22.655700+0000","last_fullsized":"2024-03-29T04:09:22.655700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.323173+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.322174+0000","last_became_peered":"2024-03-29T04:09:16.322174+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329721+0000","last_change":"2024-03-29T04:09:20.341688+0000","last_active":"2024-03-29T04:09:22.329721+0000","last_peered":"2024-03-29T04:09:22.329721+0000","last_clean":"2024-03-29T04:09:22.329721+0000","last_became_active":"2024-03-29T04:09:20.341328+0000","last_became_peered":"2024-03-29T04:09:20.341328+0000","last_unstale":"2024-03-29T04:09:22.329721+0000","last_undegraded":"2024-03-29T04:09:22.329721+0000","last_fullsized":"2024-03-29T04:09:22.329721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329443+0000","last_change":"2024-03-29T04:09:16.318229+0000","last_active":"2024-03-29T04:09:22.329443+0000","last_peered":"2024-03-29T04:09:22.329443+0000","last_clean":"2024-03-29T04:09:22.329443+0000","last_became_active":"2024-03-29T04:09:16.317967+0000","last_became_peered":"2024-03-29T04:09:16.317967+0000","last_unstale":"2024-03-29T04:09:22.329443+0000","last_undegraded":"2024-03-29T04:09:22.329443+0000","last_fullsized":"2024-03-29T04:09:22.329443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655679+0000","last_change":"2024-03-29T04:09:15.308430+0000","last_active":"2024-03-29T04:09:22.655679+0000","last_peered":"2024-03-29T04:09:22.655679+0000","last_clean":"2024-03-29T04:09:22.655679+0000","last_became_active":"2024-03-29T04:09:15.308131+0000","last_became_peered":"2024-03-29T04:09:15.308131+0000","last_unstale":"2024-03-29T04:09:22.655679+0000","last_undegraded":"2024-03-29T04:09:22.655679+0000","last_fullsized":"2024-03-29T04:09:22.655679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:36:32.401409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329454+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.329454+0000","last_peered":"2024-03-29T04:09:22.329454+0000","last_clean":"2024-03-29T04:09:22.329454+0000","last_became_active":"2024-03-29T04:09:20.347330+0000","last_became_peered":"2024-03-29T04:09:20.347330+0000","last_unstale":"2024-03-29T04:09:22.329454+0000","last_undegraded":"2024-03-29T04:09:22.329454+0000","last_fullsized":"2024-03-29T04:09:22.329454+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329091+0000","last_change":"2024-03-29T04:09:16.311370+0000","last_active":"2024-03-29T04:09:22.329091+0000","last_peered":"2024-03-29T04:09:22.329091+0000","last_clean":"2024-03-29T04:09:22.329091+0000","last_became_active":"2024-03-29T04:09:16.311106+0000","last_became_peered":"2024-03-29T04:09:16.311106+0000","last_unstale":"2024-03-29T04:09:22.329091+0000","last_undegraded":"2024-03-29T04:09:22.329091+0000","last_fullsized":"2024-03-29T04:09:22.329091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656398+0000","last_change":"2024-03-29T04:09:15.306878+0000","last_active":"2024-03-29T04:09:22.656398+0000","last_peered":"2024-03-29T04:09:22.656398+0000","last_clean":"2024-03-29T04:09:22.656398+0000","last_became_active":"2024-03-29T04:09:15.306590+0000","last_became_peered":"2024-03-29T04:09:15.306590+0000","last_unstale":"2024-03-29T04:09:22.656398+0000","last_undegraded":"2024-03-29T04:09:22.656398+0000","last_fullsized":"2024-03-29T04:09:22.656398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330041+0000","last_change":"2024-03-29T04:09:20.347808+0000","last_active":"2024-03-29T04:09:22.330041+0000","last_peered":"2024-03-29T04:09:22.330041+0000","last_clean":"2024-03-29T04:09:22.330041+0000","last_became_active":"2024-03-29T04:09:20.347419+0000","last_became_peered":"2024-03-29T04:09:20.347419+0000","last_unstale":"2024-03-29T04:09:22.330041+0000","last_undegraded":"2024-03-29T04:09:22.330041+0000","last_fullsized":"2024-03-29T04:09:22.330041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:58.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328945+0000","last_change":"2024-03-29T04:09:16.309708+0000","last_active":"2024-03-29T04:09:22.328945+0000","last_peered":"2024-03-29T04:09:22.328945+0000","last_clean":"2024-03-29T04:09:22.328945+0000","last_became_active":"2024-03-29T04:09:16.309388+0000","last_became_peered":"2024-03-29T04:09:16.309388+0000","last_unstale":"2024-03-29T04:09:22.328945+0000","last_undegraded":"2024-03-29T04:09:22.328945+0000","last_fullsized":"2024-03-29T04:09:22.328945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655404+0000","last_change":"2024-03-29T04:09:15.308764+0000","last_active":"2024-03-29T04:09:22.655404+0000","last_peered":"2024-03-29T04:09:22.655404+0000","last_clean":"2024-03-29T04:09:22.655404+0000","last_became_active":"2024-03-29T04:09:15.308466+0000","last_became_peered":"2024-03-29T04:09:15.308466+0000","last_unstale":"2024-03-29T04:09:22.655404+0000","last_undegraded":"2024-03-29T04:09:22.655404+0000","last_fullsized":"2024-03-29T04:09:22.655404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657055+0000","last_change":"2024-03-29T04:09:20.883567+0000","last_active":"2024-03-29T04:09:22.657055+0000","last_peered":"2024-03-29T04:09:22.657055+0000","last_clean":"2024-03-29T04:09:22.657055+0000","last_became_active":"2024-03-29T04:09:20.883215+0000","last_became_peered":"2024-03-29T04:09:20.883215+0000","last_unstale":"2024-03-29T04:09:22.657055+0000","last_undegraded":"2024-03-29T04:09:22.657055+0000","last_fullsized":"2024-03-29T04:09:22.657055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330723+0000","last_change":"2024-03-29T04:09:16.307731+0000","last_active":"2024-03-29T04:09:22.330723+0000","last_peered":"2024-03-29T04:09:22.330723+0000","last_clean":"2024-03-29T04:09:22.330723+0000","last_became_active":"2024-03-29T04:09:16.307458+0000","last_became_peered":"2024-03-29T04:09:16.307458+0000","last_unstale":"2024-03-29T04:09:22.330723+0000","last_undegraded":"2024-03-29T04:09:22.330723+0000","last_fullsized":"2024-03-29T04:09:22.330723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:43.534030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654990+0000","last_change":"2024-03-29T04:09:15.306750+0000","last_active":"2024-03-29T04:09:22.654990+0000","last_peered":"2024-03-29T04:09:22.654990+0000","last_clean":"2024-03-29T04:09:22.654990+0000","last_became_active":"2024-03-29T04:09:15.306458+0000","last_became_peered":"2024-03-29T04:09:15.306458+0000","last_unstale":"2024-03-29T04:09:22.654990+0000","last_undegraded":"2024-03-29T04:09:22.654990+0000","last_fullsized":"2024-03-29T04:09:22.654990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330722+0000","last_change":"2024-03-29T04:09:20.346709+0000","last_active":"2024-03-29T04:09:22.330722+0000","last_peered":"2024-03-29T04:09:22.330722+0000","last_clean":"2024-03-29T04:09:22.330722+0000","last_became_active":"2024-03-29T04:09:20.346366+0000","last_became_peered":"2024-03-29T04:09:20.346366+0000","last_unstale":"2024-03-29T04:09:22.330722+0000","last_undegraded":"2024-03-29T04:09:22.330722+0000","last_fullsized":"2024-03-29T04:09:22.330722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656086+0000","last_change":"2024-03-29T04:09:16.311388+0000","last_active":"2024-03-29T04:09:22.656086+0000","last_peered":"2024-03-29T04:09:22.656086+0000","last_clean":"2024-03-29T04:09:22.656086+0000","last_became_active":"2024-03-29T04:09:16.311119+0000","last_became_peered":"2024-03-29T04:09:16.311119+0000","last_unstale":"2024-03-29T04:09:22.656086+0000","last_undegraded":"2024-03-29T04:09:22.656086+0000","last_fullsized":"2024-03-29T04:09:22.656086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330110+0000","last_change":"2024-03-29T04:09:15.301237+0000","last_active":"2024-03-29T04:09:22.330110+0000","last_peered":"2024-03-29T04:09:22.330110+0000","last_clean":"2024-03-29T04:09:22.330110+0000","last_became_active":"2024-03-29T04:09:15.300671+0000","last_became_peered":"2024-03-29T04:09:15.300671+0000","last_unstale":"2024-03-29T04:09:22.330110+0000","last_undegraded":"2024-03-29T04:09:22.330110+0000","last_fullsized":"2024-03-29T04:09:22.330110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330650+0000","last_change":"2024-03-29T04:09:20.882096+0000","last_active":"2024-03-29T04:09:22.330650+0000","last_peered":"2024-03-29T04:09:22.330650+0000","last_clean":"2024-03-29T04:09:22.330650+0000","last_became_active":"2024-03-29T04:09:20.881672+0000","last_became_peered":"2024-03-29T04:09:20.881672+0000","last_unstale":"2024-03-29T04:09:22.330650+0000","last_undegraded":"2024-03-29T04:09:22.330650+0000","last_fullsized":"2024-03-29T04:09:22.330650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329679+0000","last_change":"2024-03-29T04:09:16.308204+0000","last_active":"2024-03-29T04:09:22.329679+0000","last_peered":"2024-03-29T04:09:22.329679+0000","last_clean":"2024-03-29T04:09:22.329679+0000","last_became_active":"2024-03-29T04:09:16.307931+0000","last_became_peered":"2024-03-29T04:09:16.307931+0000","last_unstale":"2024-03-29T04:09:22.329679+0000","last_undegraded":"2024-03-29T04:09:22.329679+0000","last_fullsized":"2024-03-29T04:09:22.329679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656940+0000","last_change":"2024-03-29T04:09:15.303590+0000","last_active":"2024-03-29T04:09:22.656940+0000","last_peered":"2024-03-29T04:09:22.656940+0000","last_clean":"2024-03-29T04:09:22.656940+0000","last_became_active":"2024-03-29T04:09:15.303295+0000","last_became_peered":"2024-03-29T04:09:15.303295+0000","last_unstale":"2024-03-29T04:09:22.656940+0000","last_undegraded":"2024-03-29T04:09:22.656940+0000","last_fullsized":"2024-03-29T04:09:22.656940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328789+0000","last_change":"2024-03-29T04:09:20.884166+0000","last_active":"2024-03-29T04:09:22.328789+0000","last_peered":"2024-03-29T04:09:22.328789+0000","last_clean":"2024-03-29T04:09:22.328789+0000","last_became_active":"2024-03-29T04:09:20.883801+0000","last_became_peered":"2024-03-29T04:09:20.883801+0000","last_unstale":"2024-03-29T04:09:22.328789+0000","last_undegraded":"2024-03-29T04:09:22.328789+0000","last_fullsized":"2024-03-29T04:09:22.328789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:45.424351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656397+0000","last_change":"2024-03-29T04:09:16.307299+0000","last_active":"2024-03-29T04:09:22.656397+0000","last_peered":"2024-03-29T04:09:22.656397+0000","last_clean":"2024-03-29T04:09:22.656397+0000","last_became_active":"2024-03-29T04:09:16.306982+0000","last_became_peered":"2024-03-29T04:09:16.306982+0000","last_unstale":"2024-03-29T04:09:22.656397+0000","last_undegraded":"2024-03-29T04:09:22.656397+0000","last_fullsized":"2024-03-29T04:09:22.656397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656158+0000","last_change":"2024-03-29T04:09:15.308823+0000","last_active":"2024-03-29T04:09:22.656158+0000","last_peered":"2024-03-29T04:09:22.656158+0000","last_clean":"2024-03-29T04:09:22.656158+0000","last_became_active":"2024-03-29T04:09:15.308335+0000","last_became_peered":"2024-03-29T04:09:15.308335+0000","last_unstale":"2024-03-29T04:09:22.656158+0000","last_undegraded":"2024-03-29T04:09:22.656158+0000","last_fullsized":"2024-03-29T04:09:22.656158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:09.173137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655324+0000","last_change":"2024-03-29T04:09:20.882760+0000","last_active":"2024-03-29T04:09:22.655324+0000","last_peered":"2024-03-29T04:09:22.655324+0000","last_clean":"2024-03-29T04:09:22.655324+0000","last_became_active":"2024-03-29T04:09:20.881819+0000","last_became_peered":"2024-03-29T04:09:20.881819+0000","last_unstale":"2024-03-29T04:09:22.655324+0000","last_undegraded":"2024-03-29T04:09:22.655324+0000","last_fullsized":"2024-03-29T04:09:22.655324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655417+0000","last_change":"2024-03-29T04:09:16.318039+0000","last_active":"2024-03-29T04:09:22.655417+0000","last_peered":"2024-03-29T04:09:22.655417+0000","last_clean":"2024-03-29T04:09:22.655417+0000","last_became_active":"2024-03-29T04:09:16.317784+0000","last_became_peered":"2024-03-29T04:09:16.317784+0000","last_unstale":"2024-03-29T04:09:22.655417+0000","last_undegraded":"2024-03-29T04:09:22.655417+0000","last_fullsized":"2024-03-29T04:09:22.655417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656459+0000","last_change":"2024-03-29T04:09:15.302818+0000","last_active":"2024-03-29T04:09:22.656459+0000","last_peered":"2024-03-29T04:09:22.656459+0000","last_clean":"2024-03-29T04:09:22.656459+0000","last_became_active":"2024-03-29T04:09:15.302521+0000","last_became_peered":"2024-03-29T04:09:15.302521+0000","last_unstale":"2024-03-29T04:09:22.656459+0000","last_undegraded":"2024-03-29T04:09:22.656459+0000","last_fullsized":"2024-03-29T04:09:22.656459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:12.034598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656072+0000","last_change":"2024-03-29T04:09:20.351951+0000","last_active":"2024-03-29T04:09:22.656072+0000","last_peered":"2024-03-29T04:09:22.656072+0000","last_clean":"2024-03-29T04:09:22.656072+0000","last_became_active":"2024-03-29T04:09:20.351337+0000","last_became_peered":"2024-03-29T04:09:20.351337+0000","last_unstale":"2024-03-29T04:09:22.656072+0000","last_undegraded":"2024-03-29T04:09:22.656072+0000","last_fullsized":"2024-03-29T04:09:22.656072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330495+0000","last_change":"2024-03-29T04:09:16.316074+0000","last_active":"2024-03-29T04:09:22.330495+0000","last_peered":"2024-03-29T04:09:22.330495+0000","last_clean":"2024-03-29T04:09:22.330495+0000","last_became_active":"2024-03-29T04:09:16.315644+0000","last_became_peered":"2024-03-29T04:09:16.315644+0000","last_unstale":"2024-03-29T04:09:22.330495+0000","last_undegraded":"2024-03-29T04:09:22.330495+0000","last_fullsized":"2024-03-29T04:09:22.330495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:13.041449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655320+0000","last_change":"2024-03-29T04:09:15.307707+0000","last_active":"2024-03-29T04:09:22.655320+0000","last_peered":"2024-03-29T04:09:22.655320+0000","last_clean":"2024-03-29T04:09:22.655320+0000","last_became_active":"2024-03-29T04:09:15.307411+0000","last_became_peered":"2024-03-29T04:09:15.307411+0000","last_unstale":"2024-03-29T04:09:22.655320+0000","last_undegraded":"2024-03-29T04:09:22.655320+0000","last_fullsized":"2024-03-29T04:09:22.655320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656552+0000","last_change":"2024-03-29T04:09:20.346773+0000","last_active":"2024-03-29T04:09:22.656552+0000","last_peered":"2024-03-29T04:09:22.656552+0000","last_clean":"2024-03-29T04:09:22.656552+0000","last_became_active":"2024-03-29T04:09:20.346383+0000","last_became_peered":"2024-03-29T04:09:20.346383+0000","last_unstale":"2024-03-29T04:09:22.656552+0000","last_undegraded":"2024-03-29T04:09:22.656552+0000","last_fullsized":"2024-03-29T04:09:22.656552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655287+0000","last_change":"2024-03-29T04:09:16.311447+0000","last_active":"2024-03-29T04:09:22.655287+0000","last_peered":"2024-03-29T04:09:22.655287+0000","last_clean":"2024-03-29T04:09:22.655287+0000","last_became_active":"2024-03-29T04:09:16.311171+0000","last_became_peered":"2024-03-29T04:09:16.311171+0000","last_unstale":"2024-03-29T04:09:22.655287+0000","last_undegraded":"2024-03-29T04:09:22.655287+0000","last_fullsized":"2024-03-29T04:09:22.655287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656334+0000","last_change":"2024-03-29T04:09:15.307046+0000","last_active":"2024-03-29T04:09:22.656334+0000","last_peered":"2024-03-29T04:09:22.656334+0000","last_clean":"2024-03-29T04:09:22.656334+0000","last_became_active":"2024-03-29T04:09:15.305761+0000","last_became_peered":"2024-03-29T04:09:15.305761+0000","last_unstale":"2024-03-29T04:09:22.656334+0000","last_undegraded":"2024-03-29T04:09:22.656334+0000","last_fullsized":"2024-03-29T04:09:22.656334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.657098+0000","last_change":"2024-03-29T04:09:20.346732+0000","last_active":"2024-03-29T04:09:22.657098+0000","last_peered":"2024-03-29T04:09:22.657098+0000","last_clean":"2024-03-29T04:09:22.657098+0000","last_became_active":"2024-03-29T04:09:20.346121+0000","last_became_peered":"2024-03-29T04:09:20.346121+0000","last_unstale":"2024-03-29T04:09:22.657098+0000","last_undegraded":"2024-03-29T04:09:22.657098+0000","last_fullsized":"2024-03-29T04:09:22.657098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655371+0000","last_change":"2024-03-29T04:09:16.318196+0000","last_active":"2024-03-29T04:09:22.655371+0000","last_peered":"2024-03-29T04:09:22.655371+0000","last_clean":"2024-03-29T04:09:22.655371+0000","last_became_active":"2024-03-29T04:09:16.317906+0000","last_became_peered":"2024-03-29T04:09:16.317906+0000","last_unstale":"2024-03-29T04:09:22.655371+0000","last_undegraded":"2024-03-29T04:09:22.655371+0000","last_fullsized":"2024-03-29T04:09:22.655371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:36.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655070+0000","last_change":"2024-03-29T04:09:15.308083+0000","last_active":"2024-03-29T04:09:22.655070+0000","last_peered":"2024-03-29T04:09:22.655070+0000","last_clean":"2024-03-29T04:09:22.655070+0000","last_became_active":"2024-03-29T04:09:15.307813+0000","last_became_peered":"2024-03-29T04:09:15.307813+0000","last_unstale":"2024-03-29T04:09:22.655070+0000","last_undegraded":"2024-03-29T04:09:22.655070+0000","last_fullsized":"2024-03-29T04:09:22.655070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655909+0000","last_change":"2024-03-29T04:09:20.882216+0000","last_active":"2024-03-29T04:09:22.655909+0000","last_peered":"2024-03-29T04:09:22.655909+0000","last_clean":"2024-03-29T04:09:22.655909+0000","last_became_active":"2024-03-29T04:09:20.881801+0000","last_became_peered":"2024-03-29T04:09:20.881801+0000","last_unstale":"2024-03-29T04:09:22.655909+0000","last_undegraded":"2024-03-29T04:09:22.655909+0000","last_fullsized":"2024-03-29T04:09:22.655909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:16:29.533300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328920+0000","last_change":"2024-03-29T04:09:16.319621+0000","last_active":"2024-03-29T04:09:22.328920+0000","last_peered":"2024-03-29T04:09:22.328920+0000","last_clean":"2024-03-29T04:09:22.328920+0000","last_became_active":"2024-03-29T04:09:16.318978+0000","last_became_peered":"2024-03-29T04:09:16.318978+0000","last_unstale":"2024-03-29T04:09:22.328920+0000","last_undegraded":"2024-03-29T04:09:22.328920+0000","last_fullsized":"2024-03-29T04:09:22.328920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655084+0000","last_change":"2024-03-29T04:09:15.302638+0000","last_active":"2024-03-29T04:09:22.655084+0000","last_peered":"2024-03-29T04:09:22.655084+0000","last_clean":"2024-03-29T04:09:22.655084+0000","last_became_active":"2024-03-29T04:09:15.302211+0000","last_became_peered":"2024-03-29T04:09:15.302211+0000","last_unstale":"2024-03-29T04:09:22.655084+0000","last_undegraded":"2024-03-29T04:09:22.655084+0000","last_fullsized":"2024-03-29T04:09:22.655084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655145+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:09:22.655145+0000","last_peered":"2024-03-29T04:09:22.655145+0000","last_clean":"2024-03-29T04:09:22.655145+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:09:22.655145+0000","last_undegraded":"2024-03-29T04:09:22.655145+0000","last_fullsized":"2024-03-29T04:09:22.655145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329351+0000","last_change":"2024-03-29T04:09:20.883146+0000","last_active":"2024-03-29T04:09:22.329351+0000","last_peered":"2024-03-29T04:09:22.329351+0000","last_clean":"2024-03-29T04:09:22.329351+0000","last_became_active":"2024-03-29T04:09:20.882795+0000","last_became_peered":"2024-03-29T04:09:20.882795+0000","last_unstale":"2024-03-29T04:09:22.329351+0000","last_undegraded":"2024-03-29T04:09:22.329351+0000","last_fullsized":"2024-03-29T04:09:22.329351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:27.037555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655937+0000","last_change":"2024-03-29T04:09:16.316245+0000","last_active":"2024-03-29T04:09:22.655937+0000","last_peered":"2024-03-29T04:09:22.655937+0000","last_clean":"2024-03-29T04:09:22.655937+0000","last_became_active":"2024-03-29T04:09:16.315972+0000","last_became_peered":"2024-03-29T04:09:16.315972+0000","last_unstale":"2024-03-29T04:09:22.655937+0000","last_undegraded":"2024-03-29T04:09:22.655937+0000","last_fullsized":"2024-03-29T04:09:22.655937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656723+0000","last_change":"2024-03-29T04:09:15.304758+0000","last_active":"2024-03-29T04:09:22.656723+0000","last_peered":"2024-03-29T04:09:22.656723+0000","last_clean":"2024-03-29T04:09:22.656723+0000","last_became_active":"2024-03-29T04:09:15.304450+0000","last_became_peered":"2024-03-29T04:09:15.304450+0000","last_unstale":"2024-03-29T04:09:22.656723+0000","last_undegraded":"2024-03-29T04:09:22.656723+0000","last_fullsized":"2024-03-29T04:09:22.656723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329391+0000","last_change":"2024-03-29T04:09:20.882224+0000","last_active":"2024-03-29T04:09:22.329391+0000","last_peered":"2024-03-29T04:09:22.329391+0000","last_clean":"2024-03-29T04:09:22.329391+0000","last_became_active":"2024-03-29T04:09:20.881813+0000","last_became_peered":"2024-03-29T04:09:20.881813+0000","last_unstale":"2024-03-29T04:09:22.329391+0000","last_undegraded":"2024-03-29T04:09:22.329391+0000","last_fullsized":"2024-03-29T04:09:22.329391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:23.738232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655284+0000","last_change":"2024-03-29T04:09:16.322983+0000","last_active":"2024-03-29T04:09:22.655284+0000","last_peered":"2024-03-29T04:09:22.655284+0000","last_clean":"2024-03-29T04:09:22.655284+0000","last_became_active":"2024-03-29T04:09:16.322712+0000","last_became_peered":"2024-03-29T04:09:16.322712+0000","last_unstale":"2024-03-29T04:09:22.655284+0000","last_undegraded":"2024-03-29T04:09:22.655284+0000","last_fullsized":"2024-03-29T04:09:22.655284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:15.302835+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:15.302365+0000","last_became_peered":"2024-03-29T04:09:15.302365+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655981+0000","last_change":"2024-03-29T04:09:20.347857+0000","last_active":"2024-03-29T04:09:22.655981+0000","last_peered":"2024-03-29T04:09:22.655981+0000","last_clean":"2024-03-29T04:09:22.655981+0000","last_became_active":"2024-03-29T04:09:20.347499+0000","last_became_peered":"2024-03-29T04:09:20.347499+0000","last_unstale":"2024-03-29T04:09:22.655981+0000","last_undegraded":"2024-03-29T04:09:22.655981+0000","last_fullsized":"2024-03-29T04:09:22.655981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:58.917868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656262+0000","last_change":"2024-03-29T04:09:16.321770+0000","last_active":"2024-03-29T04:09:22.656262+0000","last_peered":"2024-03-29T04:09:22.656262+0000","last_clean":"2024-03-29T04:09:22.656262+0000","last_became_active":"2024-03-29T04:09:16.321511+0000","last_became_peered":"2024-03-29T04:09:16.321511+0000","last_unstale":"2024-03-29T04:09:22.656262+0000","last_undegraded":"2024-03-29T04:09:22.656262+0000","last_fullsized":"2024-03-29T04:09:22.656262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:16:54.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656031+0000","last_change":"2024-03-29T04:09:15.306618+0000","last_active":"2024-03-29T04:09:22.656031+0000","last_peered":"2024-03-29T04:09:22.656031+0000","last_clean":"2024-03-29T04:09:22.656031+0000","last_became_active":"2024-03-29T04:09:15.306337+0000","last_became_peered":"2024-03-29T04:09:15.306337+0000","last_unstale":"2024-03-29T04:09:22.656031+0000","last_undegraded":"2024-03-29T04:09:22.656031+0000","last_fullsized":"2024-03-29T04:09:22.656031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654617+0000","last_change":"2024-03-29T04:09:20.884325+0000","last_active":"2024-03-29T04:09:22.654617+0000","last_peered":"2024-03-29T04:09:22.654617+0000","last_clean":"2024-03-29T04:09:22.654617+0000","last_became_active":"2024-03-29T04:09:20.883509+0000","last_became_peered":"2024-03-29T04:09:20.883509+0000","last_unstale":"2024-03-29T04:09:22.654617+0000","last_undegraded":"2024-03-29T04:09:22.654617+0000","last_fullsized":"2024-03-29T04:09:22.654617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:45:02.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655965+0000","last_change":"2024-03-29T04:09:16.309527+0000","last_active":"2024-03-29T04:09:22.655965+0000","last_peered":"2024-03-29T04:09:22.655965+0000","last_clean":"2024-03-29T04:09:22.655965+0000","last_became_active":"2024-03-29T04:09:16.309219+0000","last_became_peered":"2024-03-29T04:09:16.309219+0000","last_unstale":"2024-03-29T04:09:22.655965+0000","last_undegraded":"2024-03-29T04:09:22.655965+0000","last_fullsized":"2024-03-29T04:09:22.655965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:22.997954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329782+0000","last_change":"2024-03-29T04:09:15.306652+0000","last_active":"2024-03-29T04:09:22.329782+0000","last_peered":"2024-03-29T04:09:22.329782+0000","last_clean":"2024-03-29T04:09:22.329782+0000","last_became_active":"2024-03-29T04:09:15.306354+0000","last_became_peered":"2024-03-29T04:09:15.306354+0000","last_unstale":"2024-03-29T04:09:22.329782+0000","last_undegraded":"2024-03-29T04:09:22.329782+0000","last_fullsized":"2024-03-29T04:09:22.329782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:11.111401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330540+0000","last_change":"2024-03-29T04:09:20.346199+0000","last_active":"2024-03-29T04:09:22.330540+0000","last_peered":"2024-03-29T04:09:22.330540+0000","last_clean":"2024-03-29T04:09:22.330540+0000","last_became_active":"2024-03-29T04:09:20.345826+0000","last_became_peered":"2024-03-29T04:09:20.345826+0000","last_unstale":"2024-03-29T04:09:22.330540+0000","last_undegraded":"2024-03-29T04:09:22.330540+0000","last_fullsized":"2024-03-29T04:09:22.330540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.321785+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.321533+0000","last_became_peered":"2024-03-29T04:09:16.321533+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655385+0000","last_change":"2024-03-29T04:09:15.307664+0000","last_active":"2024-03-29T04:09:22.655385+0000","last_peered":"2024-03-29T04:09:22.655385+0000","last_clean":"2024-03-29T04:09:22.655385+0000","last_became_active":"2024-03-29T04:09:15.307412+0000","last_became_peered":"2024-03-29T04:09:15.307412+0000","last_unstale":"2024-03-29T04:09:22.655385+0000","last_undegraded":"2024-03-29T04:09:22.655385+0000","last_fullsized":"2024-03-29T04:09:22.655385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:54.162503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655742+0000","last_change":"2024-03-29T04:09:20.346476+0000","last_active":"2024-03-29T04:09:22.655742+0000","last_peered":"2024-03-29T04:09:22.655742+0000","last_clean":"2024-03-29T04:09:22.655742+0000","last_became_active":"2024-03-29T04:09:20.346140+0000","last_became_peered":"2024-03-29T04:09:20.346140+0000","last_unstale":"2024-03-29T04:09:22.655742+0000","last_undegraded":"2024-03-29T04:09:22.655742+0000","last_fullsized":"2024-03-29T04:09:22.655742+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:06:59.660598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329068+0000","last_change":"2024-03-29T04:09:16.313068+0000","last_active":"2024-03-29T04:09:22.329068+0000","last_peered":"2024-03-29T04:09:22.329068+0000","last_clean":"2024-03-29T04:09:22.329068+0000","last_became_active":"2024-03-29T04:09:16.312754+0000","last_became_peered":"2024-03-29T04:09:16.312754+0000","last_unstale":"2024-03-29T04:09:22.329068+0000","last_undegraded":"2024-03-29T04:09:22.329068+0000","last_fullsized":"2024-03-29T04:09:22.329068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329238+0000","last_change":"2024-03-29T04:09:15.300287+0000","last_active":"2024-03-29T04:09:22.329238+0000","last_peered":"2024-03-29T04:09:22.329238+0000","last_clean":"2024-03-29T04:09:22.329238+0000","last_became_active":"2024-03-29T04:09:15.299961+0000","last_became_peered":"2024-03-29T04:09:15.299961+0000","last_unstale":"2024-03-29T04:09:22.329238+0000","last_undegraded":"2024-03-29T04:09:22.329238+0000","last_fullsized":"2024-03-29T04:09:22.329238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329627+0000","last_change":"2024-03-29T04:09:20.883520+0000","last_active":"2024-03-29T04:09:22.329627+0000","last_peered":"2024-03-29T04:09:22.329627+0000","last_clean":"2024-03-29T04:09:22.329627+0000","last_became_active":"2024-03-29T04:09:20.883152+0000","last_became_peered":"2024-03-29T04:09:20.883152+0000","last_unstale":"2024-03-29T04:09:22.329627+0000","last_undegraded":"2024-03-29T04:09:22.329627+0000","last_fullsized":"2024-03-29T04:09:22.329627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:25.394492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655644+0000","last_change":"2024-03-29T04:09:16.322576+0000","last_active":"2024-03-29T04:09:22.655644+0000","last_peered":"2024-03-29T04:09:22.655644+0000","last_clean":"2024-03-29T04:09:22.655644+0000","last_became_active":"2024-03-29T04:09:16.322126+0000","last_became_peered":"2024-03-29T04:09:16.322126+0000","last_unstale":"2024-03-29T04:09:22.655644+0000","last_undegraded":"2024-03-29T04:09:22.655644+0000","last_fullsized":"2024-03-29T04:09:22.655644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655221+0000","last_change":"2024-03-29T04:09:15.306114+0000","last_active":"2024-03-29T04:09:22.655221+0000","last_peered":"2024-03-29T04:09:22.655221+0000","last_clean":"2024-03-29T04:09:22.655221+0000","last_became_active":"2024-03-29T04:09:15.305893+0000","last_became_peered":"2024-03-29T04:09:15.305893+0000","last_unstale":"2024-03-29T04:09:22.655221+0000","last_undegraded":"2024-03-29T04:09:22.655221+0000","last_fullsized":"2024-03-29T04:09:22.655221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329246+0000","last_change":"2024-03-29T04:09:20.882277+0000","last_active":"2024-03-29T04:09:22.329246+0000","last_peered":"2024-03-29T04:09:22.329246+0000","last_clean":"2024-03-29T04:09:22.329246+0000","last_became_active":"2024-03-29T04:09:20.881878+0000","last_became_peered":"2024-03-29T04:09:20.881878+0000","last_unstale":"2024-03-29T04:09:22.329246+0000","last_undegraded":"2024-03-29T04:09:22.329246+0000","last_fullsized":"2024-03-29T04:09:22.329246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:02:06.012938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655785+0000","last_change":"2024-03-29T04:09:16.307882+0000","last_active":"2024-03-29T04:09:22.655785+0000","last_peered":"2024-03-29T04:09:22.655785+0000","last_clean":"2024-03-29T04:09:22.655785+0000","last_became_active":"2024-03-29T04:09:16.307620+0000","last_became_peered":"2024-03-29T04:09:16.307620+0000","last_unstale":"2024-03-29T04:09:22.655785+0000","last_undegraded":"2024-03-29T04:09:22.655785+0000","last_fullsized":"2024-03-29T04:09:22.655785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:13:34.384227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656164+0000","last_change":"2024-03-29T04:09:15.300985+0000","last_active":"2024-03-29T04:09:22.656164+0000","last_peered":"2024-03-29T04:09:22.656164+0000","last_clean":"2024-03-29T04:09:22.656164+0000","last_became_active":"2024-03-29T04:09:15.300603+0000","last_became_peered":"2024-03-29T04:09:15.300603+0000","last_unstale":"2024-03-29T04:09:22.656164+0000","last_undegraded":"2024-03-29T04:09:22.656164+0000","last_fullsized":"2024-03-29T04:09:22.656164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:12.516429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655363+0000","last_change":"2024-03-29T04:09:20.883953+0000","last_active":"2024-03-29T04:09:22.655363+0000","last_peered":"2024-03-29T04:09:22.655363+0000","last_clean":"2024-03-29T04:09:22.655363+0000","last_became_active":"2024-03-29T04:09:20.883597+0000","last_became_peered":"2024-03-29T04:09:20.883597+0000","last_unstale":"2024-03-29T04:09:22.655363+0000","last_undegraded":"2024-03-29T04:09:22.655363+0000","last_fullsized":"2024-03-29T04:09:22.655363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:27.295430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655864+0000","last_change":"2024-03-29T04:09:16.306905+0000","last_active":"2024-03-29T04:09:22.655864+0000","last_peered":"2024-03-29T04:09:22.655864+0000","last_clean":"2024-03-29T04:09:22.655864+0000","last_became_active":"2024-03-29T04:09:16.306156+0000","last_became_peered":"2024-03-29T04:09:16.306156+0000","last_unstale":"2024-03-29T04:09:22.655864+0000","last_undegraded":"2024-03-29T04:09:22.655864+0000","last_fullsized":"2024-03-29T04:09:22.655864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:56.829890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:25.831236+0000","last_change":"2024-03-29T04:09:15.302359+0000","last_active":"2024-03-29T04:09:25.831236+0000","last_peered":"2024-03-29T04:09:25.831236+0000","last_clean":"2024-03-29T04:09:25.831236+0000","last_became_active":"2024-03-29T04:09:15.302044+0000","last_became_peered":"2024-03-29T04:09:15.302044+0000","last_unstale":"2024-03-29T04:09:25.831236+0000","last_undegraded":"2024-03-29T04:09:25.831236+0000","last_fullsized":"2024-03-29T04:09:25.831236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656202+0000","last_change":"2024-03-29T04:09:20.352101+0000","last_active":"2024-03-29T04:09:22.656202+0000","last_peered":"2024-03-29T04:09:22.656202+0000","last_clean":"2024-03-29T04:09:22.656202+0000","last_became_active":"2024-03-29T04:09:20.351597+0000","last_became_peered":"2024-03-29T04:09:20.351597+0000","last_unstale":"2024-03-29T04:09:22.656202+0000","last_undegraded":"2024-03-29T04:09:22.656202+0000","last_fullsized":"2024-03-29T04:09:22.656202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330245+0000","last_change":"2024-03-29T04:09:16.305800+0000","last_active":"2024-03-29T04:09:22.330245+0000","last_peered":"2024-03-29T04:09:22.330245+0000","last_clean":"2024-03-29T04:09:22.330245+0000","last_became_active":"2024-03-29T04:09:16.305534+0000","last_became_peered":"2024-03-29T04:09:16.305534+0000","last_unstale":"2024-03-29T04:09:22.330245+0000","last_undegraded":"2024-03-29T04:09:22.330245+0000","last_fullsized":"2024-03-29T04:09:22.330245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655674+0000","last_change":"2024-03-29T04:09:15.309193+0000","last_active":"2024-03-29T04:09:22.655674+0000","last_peered":"2024-03-29T04:09:22.655674+0000","last_clean":"2024-03-29T04:09:22.655674+0000","last_became_active":"2024-03-29T04:09:15.308884+0000","last_became_peered":"2024-03-29T04:09:15.308884+0000","last_unstale":"2024-03-29T04:09:22.655674+0000","last_undegraded":"2024-03-29T04:09:22.655674+0000","last_fullsized":"2024-03-29T04:09:22.655674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:42.527179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330135+0000","last_change":"2024-03-29T04:09:20.348029+0000","last_active":"2024-03-29T04:09:22.330135+0000","last_peered":"2024-03-29T04:09:22.330135+0000","last_clean":"2024-03-29T04:09:22.330135+0000","last_became_active":"2024-03-29T04:09:20.347196+0000","last_became_peered":"2024-03-29T04:09:20.347196+0000","last_unstale":"2024-03-29T04:09:22.330135+0000","last_undegraded":"2024-03-29T04:09:22.330135+0000","last_fullsized":"2024-03-29T04:09:22.330135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:03.628267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656194+0000","last_change":"2024-03-29T04:09:16.318939+0000","last_active":"2024-03-29T04:09:22.656194+0000","last_peered":"2024-03-29T04:09:22.656194+0000","last_clean":"2024-03-29T04:09:22.656194+0000","last_became_active":"2024-03-29T04:09:16.318327+0000","last_became_peered":"2024-03-29T04:09:16.318327+0000","last_unstale":"2024-03-29T04:09:22.656194+0000","last_undegraded":"2024-03-29T04:09:22.656194+0000","last_fullsized":"2024-03-29T04:09:22.656194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654888+0000","last_change":"2024-03-29T04:09:15.300878+0000","last_active":"2024-03-29T04:09:22.654888+0000","last_peered":"2024-03-29T04:09:22.654888+0000","last_clean":"2024-03-29T04:09:22.654888+0000","last_became_active":"2024-03-29T04:09:15.300567+0000","last_became_peered":"2024-03-29T04:09:15.300567+0000","last_unstale":"2024-03-29T04:09:22.654888+0000","last_undegraded":"2024-03-29T04:09:22.654888+0000","last_fullsized":"2024-03-29T04:09:22.654888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:15.549158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328959+0000","last_change":"2024-03-29T04:09:20.343510+0000","last_active":"2024-03-29T04:09:22.328959+0000","last_peered":"2024-03-29T04:09:22.328959+0000","last_clean":"2024-03-29T04:09:22.328959+0000","last_became_active":"2024-03-29T04:09:20.343070+0000","last_became_peered":"2024-03-29T04:09:20.343070+0000","last_unstale":"2024-03-29T04:09:22.328959+0000","last_undegraded":"2024-03-29T04:09:22.328959+0000","last_fullsized":"2024-03-29T04:09:22.328959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:10.010570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654868+0000","last_change":"2024-03-29T04:09:16.314479+0000","last_active":"2024-03-29T04:09:22.654868+0000","last_peered":"2024-03-29T04:09:22.654868+0000","last_clean":"2024-03-29T04:09:22.654868+0000","last_became_active":"2024-03-29T04:09:16.313990+0000","last_became_peered":"2024-03-29T04:09:16.313990+0000","last_unstale":"2024-03-29T04:09:22.654868+0000","last_undegraded":"2024-03-29T04:09:22.654868+0000","last_fullsized":"2024-03-29T04:09:22.654868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:48:53.428682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329948+0000","last_change":"2024-03-29T04:09:15.304665+0000","last_active":"2024-03-29T04:09:22.329948+0000","last_peered":"2024-03-29T04:09:22.329948+0000","last_clean":"2024-03-29T04:09:22.329948+0000","last_became_active":"2024-03-29T04:09:15.304381+0000","last_became_peered":"2024-03-29T04:09:15.304381+0000","last_unstale":"2024-03-29T04:09:22.329948+0000","last_undegraded":"2024-03-29T04:09:22.329948+0000","last_fullsized":"2024-03-29T04:09:22.329948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:17.595000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329035+0000","last_change":"2024-03-29T04:09:20.884477+0000","last_active":"2024-03-29T04:09:22.329035+0000","last_peered":"2024-03-29T04:09:22.329035+0000","last_clean":"2024-03-29T04:09:22.329035+0000","last_became_active":"2024-03-29T04:09:20.884123+0000","last_became_peered":"2024-03-29T04:09:20.884123+0000","last_unstale":"2024-03-29T04:09:22.329035+0000","last_undegraded":"2024-03-29T04:09:22.329035+0000","last_fullsized":"2024-03-29T04:09:22.329035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:11.194987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655599+0000","last_change":"2024-03-29T04:09:20.883017+0000","last_active":"2024-03-29T04:09:22.655599+0000","last_peered":"2024-03-29T04:09:22.655599+0000","last_clean":"2024-03-29T04:09:22.655599+0000","last_became_active":"2024-03-29T04:09:20.882655+0000","last_became_peered":"2024-03-29T04:09:20.882655+0000","last_unstale":"2024-03-29T04:09:22.655599+0000","last_undegraded":"2024-03-29T04:09:22.655599+0000","last_fullsized":"2024-03-29T04:09:22.655599+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:30.663157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655207+0000","last_change":"2024-03-29T04:09:16.313418+0000","last_active":"2024-03-29T04:09:22.655207+0000","last_peered":"2024-03-29T04:09:22.655207+0000","last_clean":"2024-03-29T04:09:22.655207+0000","last_became_active":"2024-03-29T04:09:16.312923+0000","last_became_peered":"2024-03-29T04:09:16.312923+0000","last_unstale":"2024-03-29T04:09:22.655207+0000","last_undegraded":"2024-03-29T04:09:22.655207+0000","last_fullsized":"2024-03-29T04:09:22.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654732+0000","last_change":"2024-03-29T04:09:15.304449+0000","last_active":"2024-03-29T04:09:22.654732+0000","last_peered":"2024-03-29T04:09:22.654732+0000","last_clean":"2024-03-29T04:09:22.654732+0000","last_became_active":"2024-03-29T04:09:15.304164+0000","last_became_peered":"2024-03-29T04:09:15.304164+0000","last_unstale":"2024-03-29T04:09:22.654732+0000","last_undegraded":"2024-03-29T04:09:22.654732+0000","last_fullsized":"2024-03-29T04:09:22.654732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329921+0000","last_change":"2024-03-29T04:09:20.344669+0000","last_active":"2024-03-29T04:09:22.329921+0000","last_peered":"2024-03-29T04:09:22.329921+0000","last_clean":"2024-03-29T04:09:22.329921+0000","last_became_active":"2024-03-29T04:09:20.344359+0000","last_became_peered":"2024-03-29T04:09:20.344359+0000","last_unstale":"2024-03-29T04:09:22.329921+0000","last_undegraded":"2024-03-29T04:09:22.329921+0000","last_fullsized":"2024-03-29T04:09:22.329921+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:55.712244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328865+0000","last_change":"2024-03-29T04:09:16.308421+0000","last_active":"2024-03-29T04:09:22.328865+0000","last_peered":"2024-03-29T04:09:22.328865+0000","last_clean":"2024-03-29T04:09:22.328865+0000","last_became_active":"2024-03-29T04:09:16.308142+0000","last_became_peered":"2024-03-29T04:09:16.308142+0000","last_unstale":"2024-03-29T04:09:22.328865+0000","last_undegraded":"2024-03-29T04:09:22.328865+0000","last_fullsized":"2024-03-29T04:09:22.328865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329237+0000","last_change":"2024-03-29T04:09:15.302691+0000","last_active":"2024-03-29T04:09:22.329237+0000","last_peered":"2024-03-29T04:09:22.329237+0000","last_clean":"2024-03-29T04:09:22.329237+0000","last_became_active":"2024-03-29T04:09:15.302305+0000","last_became_peered":"2024-03-29T04:09:15.302305+0000","last_unstale":"2024-03-29T04:09:22.329237+0000","last_undegraded":"2024-03-29T04:09:22.329237+0000","last_fullsized":"2024-03-29T04:09:22.329237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:49.223381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329606+0000","last_change":"2024-03-29T04:09:15.299655+0000","last_active":"2024-03-29T04:09:22.329606+0000","last_peered":"2024-03-29T04:09:22.329606+0000","last_clean":"2024-03-29T04:09:22.329606+0000","last_became_active":"2024-03-29T04:09:15.299361+0000","last_became_peered":"2024-03-29T04:09:15.299361+0000","last_unstale":"2024-03-29T04:09:22.329606+0000","last_undegraded":"2024-03-29T04:09:22.329606+0000","last_fullsized":"2024-03-29T04:09:22.329606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:40.177501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656124+0000","last_change":"2024-03-29T04:09:16.322096+0000","last_active":"2024-03-29T04:09:22.656124+0000","last_peered":"2024-03-29T04:09:22.656124+0000","last_clean":"2024-03-29T04:09:22.656124+0000","last_became_active":"2024-03-29T04:09:16.321833+0000","last_became_peered":"2024-03-29T04:09:16.321833+0000","last_unstale":"2024-03-29T04:09:22.656124+0000","last_undegraded":"2024-03-29T04:09:22.656124+0000","last_fullsized":"2024-03-29T04:09:22.656124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656607+0000","last_change":"2024-03-29T04:09:20.347701+0000","last_active":"2024-03-29T04:09:22.656607+0000","last_peered":"2024-03-29T04:09:22.656607+0000","last_clean":"2024-03-29T04:09:22.656607+0000","last_became_active":"2024-03-29T04:09:20.347088+0000","last_became_peered":"2024-03-29T04:09:20.347088+0000","last_unstale":"2024-03-29T04:09:22.656607+0000","last_undegraded":"2024-03-29T04:09:22.656607+0000","last_fullsized":"2024-03-29T04:09:22.656607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:45.089535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329185+0000","last_change":"2024-03-29T04:09:15.301823+0000","last_active":"2024-03-29T04:09:22.329185+0000","last_peered":"2024-03-29T04:09:22.329185+0000","last_clean":"2024-03-29T04:09:22.329185+0000","last_became_active":"2024-03-29T04:09:15.301182+0000","last_became_peered":"2024-03-29T04:09:15.301182+0000","last_unstale":"2024-03-29T04:09:22.329185+0000","last_undegraded":"2024-03-29T04:09:22.329185+0000","last_fullsized":"2024-03-29T04:09:22.329185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:14.272468+0000","last_clean_scrub_stamp":"2024-03-29T04:09:14.272468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:45.618821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329886+0000","last_change":"2024-03-29T04:09:16.311797+0000","last_active":"2024-03-29T04:09:22.329886+0000","last_peered":"2024-03-29T04:09:22.329886+0000","last_clean":"2024-03-29T04:09:22.329886+0000","last_became_active":"2024-03-29T04:09:16.311544+0000","last_became_peered":"2024-03-29T04:09:16.311544+0000","last_unstale":"2024-03-29T04:09:22.329886+0000","last_undegraded":"2024-03-29T04:09:22.329886+0000","last_fullsized":"2024-03-29T04:09:22.329886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:41.184352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329592+0000","last_change":"2024-03-29T04:09:20.344647+0000","last_active":"2024-03-29T04:09:22.329592+0000","last_peered":"2024-03-29T04:09:22.329592+0000","last_clean":"2024-03-29T04:09:22.329592+0000","last_became_active":"2024-03-29T04:09:20.344278+0000","last_became_peered":"2024-03-29T04:09:20.344278+0000","last_unstale":"2024-03-29T04:09:22.329592+0000","last_undegraded":"2024-03-29T04:09:22.329592+0000","last_fullsized":"2024-03-29T04:09:22.329592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:19.318920+0000","last_clean_scrub_stamp":"2024-03-29T04:09:19.318920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:19:15.627313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328975+0000","last_change":"2024-03-29T04:09:16.311872+0000","last_active":"2024-03-29T04:09:22.328975+0000","last_peered":"2024-03-29T04:09:22.328975+0000","last_clean":"2024-03-29T04:09:22.328975+0000","last_became_active":"2024-03-29T04:09:16.311576+0000","last_became_peered":"2024-03-29T04:09:16.311576+0000","last_unstale":"2024-03-29T04:09:22.328975+0000","last_undegraded":"2024-03-29T04:09:22.328975+0000","last_fullsized":"2024-03-29T04:09:22.328975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656255+0000","last_change":"2024-03-29T04:09:16.315268+0000","last_active":"2024-03-29T04:09:22.656255+0000","last_peered":"2024-03-29T04:09:22.656255+0000","last_clean":"2024-03-29T04:09:22.656255+0000","last_became_active":"2024-03-29T04:09:16.315002+0000","last_became_peered":"2024-03-29T04:09:16.315002+0000","last_unstale":"2024-03-29T04:09:22.656255+0000","last_undegraded":"2024-03-29T04:09:22.656255+0000","last_fullsized":"2024-03-29T04:09:22.656255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:42.006490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.323782+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.323504+0000","last_became_peered":"2024-03-29T04:09:16.323504+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656037+0000","last_change":"2024-03-29T04:09:16.316947+0000","last_active":"2024-03-29T04:09:22.656037+0000","last_peered":"2024-03-29T04:09:22.656037+0000","last_clean":"2024-03-29T04:09:22.656037+0000","last_became_active":"2024-03-29T04:09:16.316689+0000","last_became_peered":"2024-03-29T04:09:16.316689+0000","last_unstale":"2024-03-29T04:09:22.656037+0000","last_undegraded":"2024-03-29T04:09:22.656037+0000","last_fullsized":"2024-03-29T04:09:22.656037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329288+0000","last_change":"2024-03-29T04:09:16.319962+0000","last_active":"2024-03-29T04:09:22.329288+0000","last_peered":"2024-03-29T04:09:22.329288+0000","last_clean":"2024-03-29T04:09:22.329288+0000","last_became_active":"2024-03-29T04:09:16.319316+0000","last_became_peered":"2024-03-29T04:09:16.319316+0000","last_unstale":"2024-03-29T04:09:22.329288+0000","last_undegraded":"2024-03-29T04:09:22.329288+0000","last_fullsized":"2024-03-29T04:09:22.329288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655509+0000","last_change":"2024-03-29T04:09:16.312227+0000","last_active":"2024-03-29T04:09:22.655509+0000","last_peered":"2024-03-29T04:09:22.655509+0000","last_clean":"2024-03-29T04:09:22.655509+0000","last_became_active":"2024-03-29T04:09:16.311944+0000","last_became_peered":"2024-03-29T04:09:16.311944+0000","last_unstale":"2024-03-29T04:09:22.655509+0000","last_undegraded":"2024-03-29T04:09:22.655509+0000","last_fullsized":"2024-03-29T04:09:22.655509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330212+0000","last_change":"2024-03-29T04:09:16.306234+0000","last_active":"2024-03-29T04:09:22.330212+0000","last_peered":"2024-03-29T04:09:22.330212+0000","last_clean":"2024-03-29T04:09:22.330212+0000","last_became_active":"2024-03-29T04:09:16.305931+0000","last_became_peered":"2024-03-29T04:09:16.305931+0000","last_unstale":"2024-03-29T04:09:22.330212+0000","last_undegraded":"2024-03-29T04:09:22.330212+0000","last_fullsized":"2024-03-29T04:09:22.330212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329471+0000","last_change":"2024-03-29T04:09:16.310048+0000","last_active":"2024-03-29T04:09:22.329471+0000","last_peered":"2024-03-29T04:09:22.329471+0000","last_clean":"2024-03-29T04:09:22.329471+0000","last_became_active":"2024-03-29T04:09:16.309626+0000","last_became_peered":"2024-03-29T04:09:16.309626+0000","last_unstale":"2024-03-29T04:09:22.329471+0000","last_undegraded":"2024-03-29T04:09:22.329471+0000","last_fullsized":"2024-03-29T04:09:22.329471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654709+0000","last_change":"2024-03-29T04:09:16.324820+0000","last_active":"2024-03-29T04:09:22.654709+0000","last_peered":"2024-03-29T04:09:22.654709+0000","last_clean":"2024-03-29T04:09:22.654709+0000","last_became_active":"2024-03-29T04:09:16.324516+0000","last_became_peered":"2024-03-29T04:09:16.324516+0000","last_unstale":"2024-03-29T04:09:22.654709+0000","last_undegraded":"2024-03-29T04:09:22.654709+0000","last_fullsized":"2024-03-29T04:09:22.654709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:26:38.914807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329113+0000","last_change":"2024-03-29T04:09:16.318107+0000","last_active":"2024-03-29T04:09:22.329113+0000","last_peered":"2024-03-29T04:09:22.329113+0000","last_clean":"2024-03-29T04:09:22.329113+0000","last_became_active":"2024-03-29T04:09:16.317633+0000","last_became_peered":"2024-03-29T04:09:16.317633+0000","last_unstale":"2024-03-29T04:09:22.329113+0000","last_undegraded":"2024-03-29T04:09:22.329113+0000","last_fullsized":"2024-03-29T04:09:22.329113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654968+0000","last_change":"2024-03-29T04:09:16.317756+0000","last_active":"2024-03-29T04:09:22.654968+0000","last_peered":"2024-03-29T04:09:22.654968+0000","last_clean":"2024-03-29T04:09:22.654968+0000","last_became_active":"2024-03-29T04:09:16.317411+0000","last_became_peered":"2024-03-29T04:09:16.317411+0000","last_unstale":"2024-03-29T04:09:22.654968+0000","last_undegraded":"2024-03-29T04:09:22.654968+0000","last_fullsized":"2024-03-29T04:09:22.654968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:39:16.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655375+0000","last_change":"2024-03-29T04:09:16.314033+0000","last_active":"2024-03-29T04:09:22.655375+0000","last_peered":"2024-03-29T04:09:22.655375+0000","last_clean":"2024-03-29T04:09:22.655375+0000","last_became_active":"2024-03-29T04:09:16.313440+0000","last_became_peered":"2024-03-29T04:09:16.313440+0000","last_unstale":"2024-03-29T04:09:22.655375+0000","last_undegraded":"2024-03-29T04:09:22.655375+0000","last_fullsized":"2024-03-29T04:09:22.655375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329544+0000","last_change":"2024-03-29T04:09:16.318843+0000","last_active":"2024-03-29T04:09:22.329544+0000","last_peered":"2024-03-29T04:09:22.329544+0000","last_clean":"2024-03-29T04:09:22.329544+0000","last_became_active":"2024-03-29T04:09:16.318563+0000","last_became_peered":"2024-03-29T04:09:16.318563+0000","last_unstale":"2024-03-29T04:09:22.329544+0000","last_undegraded":"2024-03-29T04:09:22.329544+0000","last_fullsized":"2024-03-29T04:09:22.329544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656080+0000","last_change":"2024-03-29T04:09:16.313733+0000","last_active":"2024-03-29T04:09:22.656080+0000","last_peered":"2024-03-29T04:09:22.656080+0000","last_clean":"2024-03-29T04:09:22.656080+0000","last_became_active":"2024-03-29T04:09:16.313193+0000","last_became_peered":"2024-03-29T04:09:16.313193+0000","last_unstale":"2024-03-29T04:09:22.656080+0000","last_undegraded":"2024-03-29T04:09:22.656080+0000","last_fullsized":"2024-03-29T04:09:22.656080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:59.588666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655943+0000","last_change":"2024-03-29T04:09:16.300679+0000","last_active":"2024-03-29T04:09:22.655943+0000","last_peered":"2024-03-29T04:09:22.655943+0000","last_clean":"2024-03-29T04:09:22.655943+0000","last_became_active":"2024-03-29T04:09:16.300399+0000","last_became_peered":"2024-03-29T04:09:16.300399+0000","last_unstale":"2024-03-29T04:09:22.655943+0000","last_undegraded":"2024-03-29T04:09:22.655943+0000","last_fullsized":"2024-03-29T04:09:22.655943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:10.179988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655462+0000","last_change":"2024-03-29T04:09:16.314857+0000","last_active":"2024-03-29T04:09:22.655462+0000","last_peered":"2024-03-29T04:09:22.655462+0000","last_clean":"2024-03-29T04:09:22.655462+0000","last_became_active":"2024-03-29T04:09:16.314478+0000","last_became_peered":"2024-03-29T04:09:16.314478+0000","last_unstale":"2024-03-29T04:09:22.655462+0000","last_undegraded":"2024-03-29T04:09:22.655462+0000","last_fullsized":"2024-03-29T04:09:22.655462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:06:51.672643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654626+0000","last_change":"2024-03-29T04:09:16.314396+0000","last_active":"2024-03-29T04:09:22.654626+0000","last_peered":"2024-03-29T04:09:22.654626+0000","last_clean":"2024-03-29T04:09:22.654626+0000","last_became_active":"2024-03-29T04:09:16.313977+0000","last_became_peered":"2024-03-29T04:09:16.313977+0000","last_unstale":"2024-03-29T04:09:22.654626+0000","last_undegraded":"2024-03-29T04:09:22.654626+0000","last_fullsized":"2024-03-29T04:09:22.654626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:51.619092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655312+0000","last_change":"2024-03-29T04:09:16.320206+0000","last_active":"2024-03-29T04:09:22.655312+0000","last_peered":"2024-03-29T04:09:22.655312+0000","last_clean":"2024-03-29T04:09:22.655312+0000","last_became_active":"2024-03-29T04:09:16.319909+0000","last_became_peered":"2024-03-29T04:09:16.319909+0000","last_unstale":"2024-03-29T04:09:22.655312+0000","last_undegraded":"2024-03-29T04:09:22.655312+0000","last_fullsized":"2024-03-29T04:09:22.655312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:38:31.917074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655765+0000","last_change":"2024-03-29T04:09:16.308762+0000","last_active":"2024-03-29T04:09:22.655765+0000","last_peered":"2024-03-29T04:09:22.655765+0000","last_clean":"2024-03-29T04:09:22.655765+0000","last_became_active":"2024-03-29T04:09:16.308446+0000","last_became_peered":"2024-03-29T04:09:16.308446+0000","last_unstale":"2024-03-29T04:09:22.655765+0000","last_undegraded":"2024-03-29T04:09:22.655765+0000","last_fullsized":"2024-03-29T04:09:22.655765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:07:27.406089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328992+0000","last_change":"2024-03-29T04:09:16.306843+0000","last_active":"2024-03-29T04:09:22.328992+0000","last_peered":"2024-03-29T04:09:22.328992+0000","last_clean":"2024-03-29T04:09:22.328992+0000","last_became_active":"2024-03-29T04:09:16.306402+0000","last_became_peered":"2024-03-29T04:09:16.306402+0000","last_unstale":"2024-03-29T04:09:22.328992+0000","last_undegraded":"2024-03-29T04:09:22.328992+0000","last_fullsized":"2024-03-29T04:09:22.328992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:26.934140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330172+0000","last_change":"2024-03-29T04:09:16.310846+0000","last_active":"2024-03-29T04:09:22.330172+0000","last_peered":"2024-03-29T04:09:22.330172+0000","last_clean":"2024-03-29T04:09:22.330172+0000","last_became_active":"2024-03-29T04:09:16.309820+0000","last_became_peered":"2024-03-29T04:09:16.309820+0000","last_unstale":"2024-03-29T04:09:22.330172+0000","last_undegraded":"2024-03-29T04:09:22.330172+0000","last_fullsized":"2024-03-29T04:09:22.330172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:47:18.601851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.329788+0000","last_change":"2024-03-29T04:09:16.316170+0000","last_active":"2024-03-29T04:09:22.329788+0000","last_peered":"2024-03-29T04:09:22.329788+0000","last_clean":"2024-03-29T04:09:22.329788+0000","last_became_active":"2024-03-29T04:09:16.315896+0000","last_became_peered":"2024-03-29T04:09:16.315896+0000","last_unstale":"2024-03-29T04:09:22.329788+0000","last_undegraded":"2024-03-29T04:09:22.329788+0000","last_fullsized":"2024-03-29T04:09:22.329788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.330150+0000","last_change":"2024-03-29T04:09:16.316842+0000","last_active":"2024-03-29T04:09:22.330150+0000","last_peered":"2024-03-29T04:09:22.330150+0000","last_clean":"2024-03-29T04:09:22.330150+0000","last_became_active":"2024-03-29T04:09:16.316573+0000","last_became_peered":"2024-03-29T04:09:16.316573+0000","last_unstale":"2024-03-29T04:09:22.330150+0000","last_undegraded":"2024-03-29T04:09:22.330150+0000","last_fullsized":"2024-03-29T04:09:22.330150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656222+0000","last_change":"2024-03-29T04:09:16.316209+0000","last_active":"2024-03-29T04:09:22.656222+0000","last_peered":"2024-03-29T04:09:22.656222+0000","last_clean":"2024-03-29T04:09:22.656222+0000","last_became_active":"2024-03-29T04:09:16.315942+0000","last_became_peered":"2024-03-29T04:09:16.315942+0000","last_unstale":"2024-03-29T04:09:22.656222+0000","last_undegraded":"2024-03-29T04:09:22.656222+0000","last_fullsized":"2024-03-29T04:09:22.656222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:47:19.698631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.328767+0000","last_change":"2024-03-29T04:09:16.316731+0000","last_active":"2024-03-29T04:09:22.328767+0000","last_peered":"2024-03-29T04:09:22.328767+0000","last_clean":"2024-03-29T04:09:22.328767+0000","last_became_active":"2024-03-29T04:09:16.316418+0000","last_became_peered":"2024-03-29T04:09:16.316418+0000","last_unstale":"2024-03-29T04:09:22.328767+0000","last_undegraded":"2024-03-29T04:09:22.328767+0000","last_fullsized":"2024-03-29T04:09:22.328767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:35:20.896082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655047+0000","last_change":"2024-03-29T04:09:16.312822+0000","last_active":"2024-03-29T04:09:22.655047+0000","last_peered":"2024-03-29T04:09:22.655047+0000","last_clean":"2024-03-29T04:09:22.655047+0000","last_became_active":"2024-03-29T04:09:16.312495+0000","last_became_peered":"2024-03-29T04:09:16.312495+0000","last_unstale":"2024-03-29T04:09:22.655047+0000","last_undegraded":"2024-03-29T04:09:22.655047+0000","last_fullsized":"2024-03-29T04:09:22.655047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.654882+0000","last_change":"2024-03-29T04:09:16.316248+0000","last_active":"2024-03-29T04:09:22.654882+0000","last_peered":"2024-03-29T04:09:22.654882+0000","last_clean":"2024-03-29T04:09:22.654882+0000","last_became_active":"2024-03-29T04:09:16.315901+0000","last_became_peered":"2024-03-29T04:09:16.315901+0000","last_unstale":"2024-03-29T04:09:22.654882+0000","last_undegraded":"2024-03-29T04:09:22.654882+0000","last_fullsized":"2024-03-29T04:09:22.654882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:15.532970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655574+0000","last_change":"2024-03-29T04:09:16.307754+0000","last_active":"2024-03-29T04:09:22.655574+0000","last_peered":"2024-03-29T04:09:22.655574+0000","last_clean":"2024-03-29T04:09:22.655574+0000","last_became_active":"2024-03-29T04:09:16.307322+0000","last_became_peered":"2024-03-29T04:09:16.307322+0000","last_unstale":"2024-03-29T04:09:22.655574+0000","last_undegraded":"2024-03-29T04:09:22.655574+0000","last_fullsized":"2024-03-29T04:09:22.655574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:56:22.859698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655124+0000","last_change":"2024-03-29T04:09:16.323875+0000","last_active":"2024-03-29T04:09:22.655124+0000","last_peered":"2024-03-29T04:09:22.655124+0000","last_clean":"2024-03-29T04:09:22.655124+0000","last_became_active":"2024-03-29T04:09:16.323578+0000","last_became_peered":"2024-03-29T04:09:16.323578+0000","last_unstale":"2024-03-29T04:09:22.655124+0000","last_undegraded":"2024-03-29T04:09:22.655124+0000","last_fullsized":"2024-03-29T04:09:22.655124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.655497+0000","last_change":"2024-03-29T04:09:16.311321+0000","last_active":"2024-03-29T04:09:22.655497+0000","last_peered":"2024-03-29T04:09:22.655497+0000","last_clean":"2024-03-29T04:09:22.655497+0000","last_became_active":"2024-03-29T04:09:16.310990+0000","last_became_peered":"2024-03-29T04:09:16.310990+0000","last_unstale":"2024-03-29T04:09:22.655497+0000","last_undegraded":"2024-03-29T04:09:22.655497+0000","last_fullsized":"2024-03-29T04:09:22.655497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:29.478844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656870+0000","last_change":"2024-03-29T04:09:16.313513+0000","last_active":"2024-03-29T04:09:22.656870+0000","last_peered":"2024-03-29T04:09:22.656870+0000","last_clean":"2024-03-29T04:09:22.656870+0000","last_became_active":"2024-03-29T04:09:16.312390+0000","last_became_peered":"2024-03-29T04:09:16.312390+0000","last_unstale":"2024-03-29T04:09:22.656870+0000","last_undegraded":"2024-03-29T04:09:22.656870+0000","last_fullsized":"2024-03-29T04:09:22.656870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:28:10.358964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-29T04:09:22.656162+0000","last_change":"2024-03-29T04:09:16.303994+0000","last_active":"2024-03-29T04:09:22.656162+0000","last_peered":"2024-03-29T04:09:22.656162+0000","last_clean":"2024-03-29T04:09:22.656162+0000","last_became_active":"2024-03-29T04:09:16.303670+0000","last_became_peered":"2024-03-29T04:09:16.303670+0000","last_unstale":"2024-03-29T04:09:22.656162+0000","last_undegraded":"2024-03-29T04:09:22.656162+0000","last_fullsized":"2024-03-29T04:09:22.656162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:15.279319+0000","last_clean_scrub_stamp":"2024-03-29T04:09:15.279319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:35.141560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:09:38.006 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:09:38.006 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T04:09:38.007 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T04:09:38.007 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T04:09:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.160+0000 7f49aed04640 1 Processor -- start 2024-03-29T04:09:38.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 -- start start 2024-03-29T04:09:38.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f49a815e950 0x7f49a80d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49a80d36f0 con 0x7f49a80d48c0 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f49a80d3cb0 con 0x7f49a815e950 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49aed04640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49a80d4d00 con 0x7f49a80d30d0 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49acaa0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f49a815e950 0x7f49a80d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49a7fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f49a80d48c0 0x7f49a80d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34684/0 (socket says 172.21.15.181:34684) 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.161+0000 7f49a7fff640 1 -- 172.21.15.181:0/550053959 learned_addr learned my addr 172.21.15.181:0/550053959 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49ad2a1640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49acaa0640 1 -- 172.21.15.181:0/550053959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 msgr2=0x7f49a80b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49acaa0640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a80b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49acaa0640 1 -- 172.21.15.181:0/550053959 >> v1:172.21.15.181:6789/0 conn(0x7f49a80d48c0 legacy=0x7f49a80d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49acaa0640 1 -- 172.21.15.181:0/550053959 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a80d39d0 con 0x7f49a815e950 2024-03-29T04:09:38.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49a77fe640 1 -- 172.21.15.181:0/550053959 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099390080 0 0) 0x7f49a80d36f0 con 0x7f49a80d48c0 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.162+0000 7f49acaa0640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80d4380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009d90 tx=0x7f49a002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb9726c2929596d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49a77fe640 1 -- 172.21.15.181:0/550053959 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0039070 con 0x7f49a815e950 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49a77fe640 1 -- 172.21.15.181:0/550053959 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a002cce0 con 0x7f49a815e950 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49a77fe640 1 -- 172.21.15.181:0/550053959 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0034660 con 0x7f49a815e950 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 -- 172.21.15.181:0/550053959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 msgr2=0x7f49a80d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80d4380 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009d90 tx=0x7f49a002bf30 comp rx=0 tx=0).stop 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 -- 172.21.15.181:0/550053959 shutdown_connections 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 --2- 172.21.15.181:0/550053959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80d4380 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 -- 172.21.15.181:0/550053959 >> 172.21.15.181:0/550053959 conn(0x7f49a80676e0 msgr2=0x7f49a8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 -- 172.21.15.181:0/550053959 shutdown_connections 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.163+0000 7f49aed04640 1 -- 172.21.15.181:0/550053959 wait complete. 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.164+0000 7f49aed04640 1 Processor -- start 2024-03-29T04:09:38.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.164+0000 7f49aed04640 1 -- start start 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 0x7f49a80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f49a815ed30 con 0x7f49a815e950 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80b7da0 con 0x7f49a80d48c0 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49aed04640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49a815eeb0 con 0x7f49a80d30d0 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49ad2a1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49ad2a1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32802/0 (socket says 172.21.15.181:32802) 2024-03-29T04:09:38.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49ad2a1640 1 -- 172.21.15.181:0/1247191430 learned_addr learned my addr 172.21.15.181:0/1247191430 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49a7fff640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 0x7f49a80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.165+0000 7f49acaa0640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 -- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 msgr2=0x7f49a8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 -- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 msgr2=0x7f49a80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 0x7f49a80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 -- 172.21.15.181:0/1247191430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a8157eb0 con 0x7f49a815e950 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49a7fff640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 0x7f49a80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49ad2a1640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f499800e930 tx=0x7f499800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998017070 con 0x7f49a815e950 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499800c980 con 0x7f49a815e950 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a815f3f0 con 0x7f49a815e950 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800cd70 con 0x7f49a815e950 2024-03-29T04:09:38.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.166+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a815f9b0 con 0x7f49a815e950 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.167+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a8075a10 con 0x7f49a815e950 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.167+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f499801a070 con 0x7f49a815e950 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.168+0000 7f49a57fa640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 0x7f499406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.168+0000 7f49acaa0640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 0x7f499406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.168+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f499808b8c0 con 0x7f49a815e950 2024-03-29T04:09:38.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.168+0000 7f49acaa0640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 0x7f499406ea20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49a0036000 tx=0x7f49a0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.172+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4998059610 con 0x7f49a815e950 2024-03-29T04:09:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.385+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f49a806b580 con 0x7f49a815e950 2024-03-29T04:09:38.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.385+0000 7f49a57fa640 1 -- 172.21.15.181:0/1247191430 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f499805d910 con 0x7f49a815e950 2024-03-29T04:09:38.365 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:38.365 INFO:teuthology.orchestra.run.smithi181.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T04:09:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.387+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 msgr2=0x7f499406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.387+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 0x7f499406ea20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f49a0036000 tx=0x7f49a0030000 comp rx=0 tx=0).stop 2024-03-29T04:09:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.387+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 msgr2=0x7f49a80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.387+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f499800e930 tx=0x7f499800edf0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 shutdown_connections 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49a815e950 0x7f49a80cdbb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49a80d48c0 0x7f49a80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f499406c560 0x7f499406ea20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 --2- 172.21.15.181:0/1247191430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49a80d30d0 0x7f49a8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 >> 172.21.15.181:0/1247191430 conn(0x7f49a80676e0 msgr2=0x7f49a815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 shutdown_connections 2024-03-29T04:09:38.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.388+0000 7f49aed04640 1 -- 172.21.15.181:0/1247191430 wait complete. 2024-03-29T04:09:38.378 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T04:09:38.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:38.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.499+0000 7f8629390640 1 Processor -- start 2024-03-29T04:09:38.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.499+0000 7f8629390640 1 -- start start 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8629390640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f862415e950 0x7f86240d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8629390640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86240d36f0 con 0x7f86240d30d0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8629390640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86240d39d0 con 0x7f86240d48c0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8629390640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f86240d3cb0 con 0x7f862415e950 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86237fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f86240d30d0 0x7f86240b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34694/0 (socket says 172.21.15.181:34694) 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86237fe640 1 -- 172.21.15.181:0/3234186521 learned_addr learned my addr 172.21.15.181:0/3234186521 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8622ffd640 1 --2- 172.21.15.181:0/3234186521 >> v2:172.21.15.187:3300/0 conn(0x7f862415e950 0x7f86240d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497404278 0 0) 0x7f86240d36f0 con 0x7f86240d30d0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8610003520 con 0x7f86240d30d0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535501757 0 0) 0x7f86240d39d0 con 0x7f86240d48c0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86240d36f0 con 0x7f86240d48c0 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8622ffd640 1 -- 172.21.15.181:0/3234186521 >> v1:172.21.15.187:6789/0 conn(0x7f86240d48c0 legacy=0x7f86240d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8622ffd640 1 -- 172.21.15.181:0/3234186521 >> v1:172.21.15.181:6789/0 conn(0x7f86240d30d0 legacy=0x7f86240b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.500+0000 7f8622ffd640 1 -- 172.21.15.181:0/3234186521 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86240b7ac0 con 0x7f862415e950 2024-03-29T04:09:38.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.501+0000 7f8622ffd640 1 --2- 172.21.15.181:0/3234186521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862415e950 0x7f86240d4380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86140097a0 tx=0x7f861402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4eb3cdcac2d19f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.501+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861402f810 con 0x7f862415e950 2024-03-29T04:09:38.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.501+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861402f9b0 con 0x7f862415e950 2024-03-29T04:09:38.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.501+0000 7f86227fc640 1 -- 172.21.15.181:0/3234186521 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861402fd60 con 0x7f862415e950 2024-03-29T04:09:38.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 -- 172.21.15.181:0/3234186521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862415e950 msgr2=0x7f86240d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 --2- 172.21.15.181:0/3234186521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862415e950 0x7f86240d4380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86140097a0 tx=0x7f861402b9f0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 -- 172.21.15.181:0/3234186521 shutdown_connections 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 --2- 172.21.15.181:0/3234186521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862415e950 0x7f86240d4380 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 -- 172.21.15.181:0/3234186521 >> 172.21.15.181:0/3234186521 conn(0x7f86240676e0 msgr2=0x7f8624067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 -- 172.21.15.181:0/3234186521 shutdown_connections 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.502+0000 7f8629390640 1 -- 172.21.15.181:0/3234186521 wait complete. 2024-03-29T04:09:38.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.503+0000 7f8629390640 1 Processor -- start 2024-03-29T04:09:38.484 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.503+0000 7f8629390640 1 -- start start 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86240d30d0 0x7f86240c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862415e950 0x7f862415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f862415ed30 con 0x7f86240d48c0 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86240d4d00 con 0x7f86240d30d0 2024-03-29T04:09:38.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f862415eeb0 con 0x7f862415e950 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32818/0 (socket says 172.21.15.181:32818) 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 -- 172.21.15.181:0/1466699930 learned_addr learned my addr 172.21.15.181:0/1466699930 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8622ffd640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86240d30d0 0x7f86240c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f86237fe640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862415e950 0x7f862415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 -- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862415e950 msgr2=0x7f862415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862415e950 0x7f862415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 -- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86240d30d0 msgr2=0x7f86240c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86240d30d0 0x7f86240c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 -- 172.21.15.181:0/1466699930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86240c9d50 con 0x7f86240d48c0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861bfff640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c500 tx=0x7f860c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0052d0 con 0x7f86240d48c0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c005490 con 0x7f86240d48c0 2024-03-29T04:09:38.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86240ca030 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c012870 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.504+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86240ca5f0 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.505+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8624075a10 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.506+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f860c012a10 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.506+0000 7f861b7fe640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 0x7f860406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.506+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f860c08b590 con 0x7f86240d48c0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.506+0000 7f8622ffd640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 0x7f860406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.507+0000 7f8622ffd640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 0x7f860406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8614036000 tx=0x7f8614008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.510+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f860c059230 con 0x7f86240d48c0 2024-03-29T04:09:38.669 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.689+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86240b9850 con 0x7f86240d48c0 2024-03-29T04:09:38.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.690+0000 7f861b7fe640 1 -- 172.21.15.181:0/1466699930 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f860c078070 con 0x7f86240d48c0 2024-03-29T04:09:38.671 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:38.671 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 msgr2=0x7f860406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 0x7f860406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8614036000 tx=0x7f8614008030 comp rx=0 tx=0).stop 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 msgr2=0x7f86240c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f860c00c500 tx=0x7f860c00c9c0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 shutdown_connections 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f860406c530 0x7f860406e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f862415e950 0x7f862415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86240d48c0 0x7f86240c9520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.673 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 --2- 172.21.15.181:0/1466699930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86240d30d0 0x7f86240c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 >> 172.21.15.181:0/1466699930 conn(0x7f86240676e0 msgr2=0x7f86240b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 shutdown_connections 2024-03-29T04:09:38.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.692+0000 7f8629390640 1 -- 172.21.15.181:0/1466699930 wait complete. 2024-03-29T04:09:38.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:38.683 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:38.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.808+0000 7ff16a60d640 1 Processor -- start 2024-03-29T04:09:38.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.808+0000 7ff16a60d640 1 -- start start 2024-03-29T04:09:38.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff16a60d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff16415e5a0 0x7ff164072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff16a60d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640d0360 0x7ff164071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff16a60d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1640b2f20 con 0x7ff1640b3fa0 2024-03-29T04:09:38.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff16a60d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ff1640b3b10 con 0x7ff16415e5a0 2024-03-29T04:09:38.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff16a60d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff16415e980 con 0x7ff1640d0360 2024-03-29T04:09:38.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff16415e5a0 0x7ff164072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff16415e5a0 0x7ff164072f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32824/0 (socket says 172.21.15.181:32824) 2024-03-29T04:09:38.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 -- 172.21.15.181:0/2812396823 learned_addr learned my addr 172.21.15.181:0/2812396823 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:38.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff168baa640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640d0360 0x7ff164071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 -- 172.21.15.181:0/2812396823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640d0360 msgr2=0x7ff164071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640d0360 0x7ff164071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 -- 172.21.15.181:0/2812396823 >> v1:172.21.15.187:6789/0 conn(0x7ff1640b3fa0 legacy=0x7ff1640cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff163fff640 1 -- 172.21.15.181:0/2812396823 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1640b31e0 con 0x7ff16415e5a0 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.809+0000 7ff162ffd640 1 -- 172.21.15.181:0/2812396823 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67662156 0 0) 0x7ff1640b2f20 con 0x7ff1640b3fa0 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.810+0000 7ff163fff640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff164072f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff154009870 tx=0x7ff15402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d9ac84a8370949 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.810+0000 7ff162ffd640 1 -- 172.21.15.181:0/2812396823 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154034070 con 0x7ff16415e5a0 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.810+0000 7ff162ffd640 1 -- 172.21.15.181:0/2812396823 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff154030840 con 0x7ff16415e5a0 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.810+0000 7ff162ffd640 1 -- 172.21.15.181:0/2812396823 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154030c10 con 0x7ff16415e5a0 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 -- 172.21.15.181:0/2812396823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 msgr2=0x7ff164072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff164072f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff154009870 tx=0x7ff15402b980 comp rx=0 tx=0).stop 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 -- 172.21.15.181:0/2812396823 shutdown_connections 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640d0360 0x7ff164071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 --2- 172.21.15.181:0/2812396823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff164072f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 -- 172.21.15.181:0/2812396823 >> 172.21.15.181:0/2812396823 conn(0x7ff1640b5b00 msgr2=0x7ff16406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 -- 172.21.15.181:0/2812396823 shutdown_connections 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.811+0000 7ff16a60d640 1 -- 172.21.15.181:0/2812396823 wait complete. 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.812+0000 7ff16a60d640 1 Processor -- start 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.812+0000 7ff16a60d640 1 -- start start 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640b3fa0 0x7ff1640cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1640d0360 0x7ff164157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff16415f990 con 0x7ff16415e5a0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640b37a0 con 0x7ff1640d0360 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff16a60d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff164073440 con 0x7ff1640b3fa0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32836/0 (socket says 172.21.15.181:32836) 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 -- 172.21.15.181:0/10154003 learned_addr learned my addr 172.21.15.181:0/10154003 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff163fff640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640b3fa0 0x7ff1640cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff1637fe640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1640d0360 0x7ff164157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 -- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640b3fa0 msgr2=0x7ff1640cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640b3fa0 0x7ff1640cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 -- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1640d0360 msgr2=0x7ff164157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1640d0360 0x7ff164157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 -- 172.21.15.181:0/10154003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1641f4930 con 0x7ff16415e5a0 2024-03-29T04:09:38.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff168baa640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00efc0 tx=0x7ff15c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.813+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c018020 con 0x7ff16415e5a0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.814+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c014480 con 0x7ff16415e5a0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.814+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1641f4c10 con 0x7ff16415e5a0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.814+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c014870 con 0x7ff16415e5a0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.814+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641f51d0 con 0x7ff16415e5a0 2024-03-29T04:09:38.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.815+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff164075890 con 0x7ff16415e5a0 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.815+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff15c005d10 con 0x7ff16415e5a0 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.815+0000 7ff160ff9640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 0x7ff15006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.815+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff15c08c710 con 0x7ff16415e5a0 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.815+0000 7ff163fff640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 0x7ff15006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.816+0000 7ff163fff640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 0x7ff15006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff154036000 tx=0x7ff15403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:38.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.820+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff15c022030 con 0x7ff16415e5a0 2024-03-29T04:09:38.974 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.993+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1640adeb0 con 0x7ff16415e5a0 2024-03-29T04:09:38.974 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.994+0000 7ff160ff9640 1 -- 172.21.15.181:0/10154003 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ff15c010040 con 0x7ff16415e5a0 2024-03-29T04:09:38.975 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 msgr2=0x7ff15006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 0x7ff15006e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff154036000 tx=0x7ff15403b040 comp rx=0 tx=0).stop 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 msgr2=0x7ff16414cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00efc0 tx=0x7ff15c00c490 comp rx=0 tx=0).stop 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 shutdown_connections 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff16415e5a0 0x7ff16414cb40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1640d0360 0x7ff164157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff15006c530 0x7ff15006e9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 --2- 172.21.15.181:0/10154003 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff1640b3fa0 0x7ff1640cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:38.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 >> 172.21.15.181:0/10154003 conn(0x7ff1640b5b00 msgr2=0x7ff1640b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:38.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 shutdown_connections 2024-03-29T04:09:38.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:38.996+0000 7ff16a60d640 1 -- 172.21.15.181:0/10154003 wait complete. 2024-03-29T04:09:38.986 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.110+0000 7f0626f9d640 1 Processor -- start 2024-03-29T04:09:39.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.110+0000 7f0626f9d640 1 -- start start 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0626f9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 0x7f062015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0626f9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 0x7f06200cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0626f9d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06200ce9c0 con 0x7f062013f0c0 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0626f9d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0620160e30 con 0x7f06201603d0 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0626f9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0620160b50 con 0x7f062006d8f0 2024-03-29T04:09:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f062553a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 0x7f06200cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0624d39640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f062013f0c0 0x7f062006d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:60606/0 (socket says 172.21.15.181:60606) 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f0624d39640 1 -- 172.21.15.181:0/4037926085 learned_addr learned my addr 172.21.15.181:0/4037926085 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 0x7f062015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061f7fe640 1 -- 172.21.15.181:0/4037926085 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40253663 0 0) 0x7f0618001490 con 0x7f062013f0c0 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061f7fe640 1 -- 172.21.15.181:0/4037926085 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f060c003520 con 0x7f062013f0c0 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061ffff640 1 -- 172.21.15.181:0/4037926085 >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 msgr2=0x7f06200cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061ffff640 1 --2- 172.21.15.181:0/4037926085 >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 0x7f06200cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061ffff640 1 -- 172.21.15.181:0/4037926085 >> v1:172.21.15.187:6789/0 conn(0x7f062013f0c0 legacy=0x7f062006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f061ffff640 1 -- 172.21.15.181:0/4037926085 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062013fa40 con 0x7f062006d8f0 2024-03-29T04:09:39.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.111+0000 7f062553a640 1 --2- 172.21.15.181:0/4037926085 >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 0x7f06200cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:39.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.112+0000 7f061ffff640 1 --2- 172.21.15.181:0/4037926085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 0x7f062015fe90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06100097a0 tx=0x7f061002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8bbeb3c0fdbba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.112+0000 7f061f7fe640 1 -- 172.21.15.181:0/4037926085 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610039070 con 0x7f062006d8f0 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.112+0000 7f061f7fe640 1 -- 172.21.15.181:0/4037926085 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06100316f0 con 0x7f062006d8f0 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.112+0000 7f061f7fe640 1 -- 172.21.15.181:0/4037926085 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610031ac0 con 0x7f062006d8f0 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 -- 172.21.15.181:0/4037926085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 msgr2=0x7f062015fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 --2- 172.21.15.181:0/4037926085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 0x7f062015fe90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06100097a0 tx=0x7f061002b790 comp rx=0 tx=0).stop 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 -- 172.21.15.181:0/4037926085 shutdown_connections 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 --2- 172.21.15.181:0/4037926085 >> v2:172.21.15.187:3300/0 conn(0x7f06201603d0 0x7f06200cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 --2- 172.21.15.181:0/4037926085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f062006d8f0 0x7f062015fe90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 -- 172.21.15.181:0/4037926085 >> 172.21.15.181:0/4037926085 conn(0x7f06200b5b00 msgr2=0x7f062006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 -- 172.21.15.181:0/4037926085 shutdown_connections 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.113+0000 7f0626f9d640 1 -- 172.21.15.181:0/4037926085 wait complete. 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.114+0000 7f0626f9d640 1 Processor -- start 2024-03-29T04:09:39.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.114+0000 7f0626f9d640 1 -- start start 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062006d8f0 0x7f06201e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06201603d0 0x7f0620131870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0620160820 con 0x7f062013f0c0 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0620140000 con 0x7f062006d8f0 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f062013f4a0 con 0x7f06201603d0 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32840/0 (socket says 172.21.15.181:32840) 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 -- 172.21.15.181:0/2934469775 learned_addr learned my addr 172.21.15.181:0/2934469775 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0624d39640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062006d8f0 0x7f06201e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f062553a640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06201603d0 0x7f0620131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 -- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06201603d0 msgr2=0x7f0620131870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06201603d0 0x7f0620131870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 -- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062006d8f0 msgr2=0x7f06201e5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062006d8f0 0x7f06201e5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 -- 172.21.15.181:0/2934469775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620141c80 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061ffff640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0610037040 tx=0x7f06100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610043070 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0610034480 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0620157880 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610039040 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.115+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0620157e40 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.117+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0620158040 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.117+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0610046070 con 0x7f062013f0c0 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.117+0000 7f061d7fa640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 0x7f060006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.117+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f06100ad3b0 con 0x7f062013f0c0 2024-03-29T04:09:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.117+0000 7f0624d39640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 0x7f060006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.118+0000 7f0624d39640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 0x7f060006e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0618005c00 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.122+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0620158040 con 0x7f062013f0c0 2024-03-29T04:09:39.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.297+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0620075890 con 0x7f062013f0c0 2024-03-29T04:09:39.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.297+0000 7f061d7fa640 1 -- 172.21.15.181:0/2934469775 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f061007afc0 con 0x7f062013f0c0 2024-03-29T04:09:39.277 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 msgr2=0x7f060006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 0x7f060006e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0618005c00 tx=0x7f0618009040 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 msgr2=0x7f0620131330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0610037040 tx=0x7f06100046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 shutdown_connections 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f060006c530 0x7f060006e9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06201603d0 0x7f0620131870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062013f0c0 0x7f0620131330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 --2- 172.21.15.181:0/2934469775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f062006d8f0 0x7f06201e5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 >> 172.21.15.181:0/2934469775 conn(0x7f06200b5b00 msgr2=0x7f06200ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 shutdown_connections 2024-03-29T04:09:39.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.299+0000 7f0626f9d640 1 -- 172.21.15.181:0/2934469775 wait complete. 2024-03-29T04:09:39.290 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:39.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 Processor -- start 2024-03-29T04:09:39.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 -- start start 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 0x7fcc4c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc4c0b66e0 con 0x7fcc4c0ba090 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fcc4c0d2d90 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.414+0000 7fcc51c74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c0d2ff0 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 0x7fcc4c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4affd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcc4c0ba090 0x7fcc4c0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34702/0 (socket says 172.21.15.181:34702) 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4affd640 1 -- 172.21.15.181:0/3415597770 learned_addr learned my addr 172.21.15.181:0/3415597770 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4bfff640 1 --2- 172.21.15.181:0/3415597770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4bfff640 1 -- 172.21.15.181:0/3415597770 >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 msgr2=0x7fcc4c0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:39.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4bfff640 1 --2- 172.21.15.181:0/3415597770 >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 0x7fcc4c0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4bfff640 1 -- 172.21.15.181:0/3415597770 >> v1:172.21.15.181:6789/0 conn(0x7fcc4c0ba090 legacy=0x7fcc4c0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4bfff640 1 -- 172.21.15.181:0/3415597770 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c0d32d0 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.415+0000 7fcc4b7fe640 1 --2- 172.21.15.181:0/3415597770 >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 0x7fcc4c0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc4bfff640 1 --2- 172.21.15.181:0/3415597770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c0b7840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009d90 tx=0x7fcc4402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c5f69a386bc727 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc4a7fc640 1 -- 172.21.15.181:0/3415597770 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44039070 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc4a7fc640 1 -- 172.21.15.181:0/3415597770 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4402cc50 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc4a7fc640 1 -- 172.21.15.181:0/3415597770 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44034440 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 -- 172.21.15.181:0/3415597770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 msgr2=0x7fcc4c0b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 --2- 172.21.15.181:0/3415597770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c0b7840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009d90 tx=0x7fcc4402bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 -- 172.21.15.181:0/3415597770 shutdown_connections 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 --2- 172.21.15.181:0/3415597770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c0b7840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 --2- 172.21.15.181:0/3415597770 >> v2:172.21.15.181:3300/0 conn(0x7fcc4c0b5dc0 0x7fcc4c0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 -- 172.21.15.181:0/3415597770 >> 172.21.15.181:0/3415597770 conn(0x7fcc4c0678f0 msgr2=0x7fcc4c0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 -- 172.21.15.181:0/3415597770 shutdown_connections 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.416+0000 7fcc51c74640 1 -- 172.21.15.181:0/3415597770 wait complete. 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.417+0000 7fcc51c74640 1 Processor -- start 2024-03-29T04:09:39.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 -- start start 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc4c0ba090 0x7fcc4c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c0d3b70 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c0d3890 con 0x7fcc4c0ba090 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.418+0000 7fcc51c74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c0d4090 con 0x7fcc4c0b73d0 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32860/0 (socket says 172.21.15.181:32860) 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 -- 172.21.15.181:0/2529367355 learned_addr learned my addr 172.21.15.181:0/2529367355 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4affd640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4bfff640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc4c0ba090 0x7fcc4c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 -- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 msgr2=0x7fcc4c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 -- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc4c0ba090 msgr2=0x7fcc4c1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc4c0ba090 0x7fcc4c1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 -- 172.21.15.181:0/2529367355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c158cf0 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc4b7fe640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00ebe0 tx=0x7fcc3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c017070 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c00cc30 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c159270 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c013470 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.419+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c15e920 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.420+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c075890 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.420+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcc3c01a070 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.421+0000 7fcc33fff640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 0x7fcc3806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.421+0000 7fcc4affd640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 0x7fcc3806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.421+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcc3c08b8c0 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.421+0000 7fcc4affd640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 0x7fcc3806e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0cae90 tx=0x7fcc40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.425+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcc3c0595e0 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.599+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc4c15efe0 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.600+0000 7fcc33fff640 1 -- 172.21.15.181:0/2529367355 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fcc3c05d8e0 con 0x7fcc4c0b5dc0 2024-03-29T04:09:39.579 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 msgr2=0x7fcc3806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 0x7fcc3806e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0cae90 tx=0x7fcc40009040 comp rx=0 tx=0).stop 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 msgr2=0x7fcc4c1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00ebe0 tx=0x7fcc3c00c520 comp rx=0 tx=0).stop 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 shutdown_connections 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc4c0ba090 0x7fcc4c1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcc3806c530 0x7fcc3806e9f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc4c0b73d0 0x7fcc4c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 --2- 172.21.15.181:0/2529367355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc4c0b5dc0 0x7fcc4c1603c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 >> 172.21.15.181:0/2529367355 conn(0x7fcc4c0678f0 msgr2=0x7fcc4c0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 shutdown_connections 2024-03-29T04:09:39.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.602+0000 7fcc51c74640 1 -- 172.21.15.181:0/2529367355 wait complete. 2024-03-29T04:09:39.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:39.695 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.716+0000 7efc7ed32640 1 Processor -- start 2024-03-29T04:09:39.696 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.716+0000 7efc7ed32640 1 -- start start 2024-03-29T04:09:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7ed32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 0x7efc7806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7ed32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 0x7efc78162830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7ed32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc780d1c10 con 0x7efc7809aa30 2024-03-29T04:09:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7ed32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7efc780d0770 con 0x7efc7809aeb0 2024-03-29T04:09:39.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7ed32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc7809b340 con 0x7efc78071020 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7d2cf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 0x7efc78162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc77fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efc7809aa30 0x7efc78162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34704/0 (socket says 172.21.15.181:34704) 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc77fff640 1 -- 172.21.15.181:0/4171335566 learned_addr learned my addr 172.21.15.181:0/4171335566 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7cace640 1 --2- 172.21.15.181:0/4171335566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 0x7efc7806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc777fe640 1 -- 172.21.15.181:0/4171335566 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945048283 0 0) 0x7efc780d1c10 con 0x7efc7809aa30 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc777fe640 1 -- 172.21.15.181:0/4171335566 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc68003520 con 0x7efc7809aa30 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7cace640 1 -- 172.21.15.181:0/4171335566 >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 msgr2=0x7efc78162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7cace640 1 --2- 172.21.15.181:0/4171335566 >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 0x7efc78162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7cace640 1 -- 172.21.15.181:0/4171335566 >> v1:172.21.15.181:6789/0 conn(0x7efc7809aa30 legacy=0x7efc78162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7cace640 1 -- 172.21.15.181:0/4171335566 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7815f180 con 0x7efc78071020 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.717+0000 7efc7d2cf640 1 --2- 172.21.15.181:0/4171335566 >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 0x7efc78162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:39.699 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.718+0000 7efc7cace640 1 --2- 172.21.15.181:0/4171335566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 0x7efc7806c840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0097a0 tx=0x7efc6c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ca3b5874d636b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.718+0000 7efc777fe640 1 -- 172.21.15.181:0/4171335566 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c035020 con 0x7efc78071020 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.718+0000 7efc777fe640 1 -- 172.21.15.181:0/4171335566 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c031830 con 0x7efc78071020 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.718+0000 7efc777fe640 1 -- 172.21.15.181:0/4171335566 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c031c00 con 0x7efc78071020 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 -- 172.21.15.181:0/4171335566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 msgr2=0x7efc7806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 --2- 172.21.15.181:0/4171335566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 0x7efc7806c840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0097a0 tx=0x7efc6c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 -- 172.21.15.181:0/4171335566 shutdown_connections 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 --2- 172.21.15.181:0/4171335566 >> v2:172.21.15.181:3300/0 conn(0x7efc7809aeb0 0x7efc78162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 --2- 172.21.15.181:0/4171335566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc78071020 0x7efc7806c840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 -- 172.21.15.181:0/4171335566 >> 172.21.15.181:0/4171335566 conn(0x7efc780b3ce0 msgr2=0x7efc780b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 -- 172.21.15.181:0/4171335566 shutdown_connections 2024-03-29T04:09:39.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.719+0000 7efc7ed32640 1 -- 172.21.15.181:0/4171335566 wait complete. 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.720+0000 7efc7ed32640 1 Processor -- start 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.720+0000 7efc7ed32640 1 -- start start 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7809aa30 0x7efc78153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc7809aeb0 0x7efc7809f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc7815fa20 con 0x7efc78071020 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc7815f740 con 0x7efc7809aa30 2024-03-29T04:09:39.703 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc78071400 con 0x7efc7809aeb0 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32878/0 (socket says 172.21.15.181:32878) 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 -- 172.21.15.181:0/2713919882 learned_addr learned my addr 172.21.15.181:0/2713919882 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7d2cf640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc7809aeb0 0x7efc7809f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc77fff640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7809aa30 0x7efc78153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 -- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc7809aeb0 msgr2=0x7efc7809f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc7809aeb0 0x7efc7809f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 -- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7809aa30 msgr2=0x7efc78153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7809aa30 0x7efc78153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 -- 172.21.15.181:0/2713919882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7809bf70 con 0x7efc78071020 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7cace640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0097a0 tx=0x7efc6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c03a830 con 0x7efc78071020 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c03a9d0 con 0x7efc78071020 2024-03-29T04:09:39.704 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7809c250 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c03adc0 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.721+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7809c810 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.722+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc78075890 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.723+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc6c04b060 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.723+0000 7efc757fa640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 0x7efc5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.723+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efc6c0ad440 con 0x7efc78071020 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.723+0000 7efc77fff640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 0x7efc5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.724+0000 7efc77fff640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 0x7efc5c06e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc780af5d0 tx=0x7efc64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.728+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc6c03f070 con 0x7efc78071020 2024-03-29T04:09:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.905+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efc780ce540 con 0x7efc78071020 2024-03-29T04:09:39.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.905+0000 7efc757fa640 1 -- 172.21.15.181:0/2713919882 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7efc6c07b690 con 0x7efc78071020 2024-03-29T04:09:39.885 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 msgr2=0x7efc5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 0x7efc5c06e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efc780af5d0 tx=0x7efc64009040 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 msgr2=0x7efc781540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0097a0 tx=0x7efc6c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 shutdown_connections 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc5c06c530 0x7efc5c06e9f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc7809aeb0 0x7efc7809f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7809aa30 0x7efc78153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 --2- 172.21.15.181:0/2713919882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc78071020 0x7efc781540f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 >> 172.21.15.181:0/2713919882 conn(0x7efc780b3ce0 msgr2=0x7efc780b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 shutdown_connections 2024-03-29T04:09:39.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:39.907+0000 7efc7ed32640 1 -- 172.21.15.181:0/2713919882 wait complete. 2024-03-29T04:09:39.898 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.018+0000 7f5d3284a640 1 Processor -- start 2024-03-29T04:09:39.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.018+0000 7f5d3284a640 1 -- start start 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d3284a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d2c0cfdd0 0x7f5d2c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d3284a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2c0b4230 con 0x7f5d2c15b1f0 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d3284a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2c0b36b0 con 0x7f5d2c0b5040 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d3284a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2c0b3990 con 0x7f5d2c0cfdd0 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d2c0cfdd0 0x7f5d2c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2bfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5d2c15b1f0 0x7f5d2c0cde20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34714/0 (socket says 172.21.15.181:34714) 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d30de7640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5d2c0b5040 0x7f5d2c15f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:60612/0 (socket says 172.21.15.181:60612) 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2bfff640 1 -- 172.21.15.181:0/1108569329 learned_addr learned my addr 172.21.15.181:0/1108569329 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:39.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2b7fe640 1 -- 172.21.15.181:0/1108569329 >> v1:172.21.15.187:6789/0 conn(0x7f5d2c0b5040 legacy=0x7f5d2c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2b7fe640 1 -- 172.21.15.181:0/1108569329 >> v1:172.21.15.181:6789/0 conn(0x7f5d2c15b1f0 legacy=0x7f5d2c0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2affd640 1 -- 172.21.15.181:0/1108569329 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149837990 0 0) 0x7f5d2c0b36b0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.019+0000 7f5d2b7fe640 1 -- 172.21.15.181:0/1108569329 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c0b3c70 con 0x7f5d2c0cfdd0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.020+0000 7f5d2b7fe640 1 --2- 172.21.15.181:0/1108569329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c0d0300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d200097a0 tx=0x7f5d2002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0c60049ab9719e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.020+0000 7f5d2affd640 1 -- 172.21.15.181:0/1108569329 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20037070 con 0x7f5d2c0cfdd0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.020+0000 7f5d2affd640 1 -- 172.21.15.181:0/1108569329 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d20030710 con 0x7f5d2c0cfdd0 2024-03-29T04:09:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.020+0000 7f5d2affd640 1 -- 172.21.15.181:0/1108569329 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20030ae0 con 0x7f5d2c0cfdd0 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 -- 172.21.15.181:0/1108569329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0cfdd0 msgr2=0x7f5d2c0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1108569329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c0d0300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d200097a0 tx=0x7f5d2002b710 comp rx=0 tx=0).stop 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 -- 172.21.15.181:0/1108569329 shutdown_connections 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1108569329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c0d0300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 -- 172.21.15.181:0/1108569329 >> 172.21.15.181:0/1108569329 conn(0x7f5d2c06b580 msgr2=0x7f5d2c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 -- 172.21.15.181:0/1108569329 shutdown_connections 2024-03-29T04:09:40.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.021+0000 7f5d3284a640 1 -- 172.21.15.181:0/1108569329 wait complete. 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.022+0000 7f5d3284a640 1 Processor -- start 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.022+0000 7f5d3284a640 1 -- start start 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d2c15b1f0 0x7f5d2c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c15b750 con 0x7f5d2c0b5040 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c15b5d0 con 0x7f5d2c0cfdd0 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2c0b3310 con 0x7f5d2c15b1f0 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32894/0 (socket says 172.21.15.181:32894) 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 -- 172.21.15.181:0/1032821232 learned_addr learned my addr 172.21.15.181:0/1032821232 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2b7fe640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d30de7640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d2c15b1f0 0x7f5d2c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 -- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d2c15b1f0 msgr2=0x7f5d2c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d2c15b1f0 0x7f5d2c0b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 -- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d2c0cfdd0 msgr2=0x7f5d2c1f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c1f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 -- 172.21.15.181:0/1032821232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c1f14d0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d2bfff640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c830 tx=0x7f5d1c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c018070 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0074e0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c007690 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c1f17b0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.023+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1f1d70 con 0x7f5d2c0b5040 2024-03-29T04:09:40.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.025+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2c075a10 con 0x7f5d2c0b5040 2024-03-29T04:09:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.025+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5d1c01b070 con 0x7f5d2c0b5040 2024-03-29T04:09:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.025+0000 7f5d28ff9640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 0x7f5d1806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.025+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5d1c08ae50 con 0x7f5d2c0b5040 2024-03-29T04:09:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.025+0000 7f5d2b7fe640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 0x7f5d1806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.026+0000 7f5d2b7fe640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 0x7f5d1806ea20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d20034000 tx=0x7f5d20008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.030+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5d1c00ce20 con 0x7f5d2c0b5040 2024-03-29T04:09:40.052 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:40.055 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:09:40.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.245+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d2c1509f0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.225 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.246+0000 7f5d28ff9640 1 -- 172.21.15.181:0/1032821232 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f5d1c058ae0 con 0x7f5d2c0b5040 2024-03-29T04:09:40.226 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:40.226 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:40.226 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:09:40.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 msgr2=0x7f5d1806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 0x7f5d1806ea20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d20034000 tx=0x7f5d20008030 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 msgr2=0x7f5d2c1f09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c830 tx=0x7f5d1c00ccf0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 shutdown_connections 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5d1806c560 0x7f5d1806ea20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d2c15b1f0 0x7f5d2c0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d2c0cfdd0 0x7f5d2c1f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 --2- 172.21.15.181:0/1032821232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d2c0b5040 0x7f5d2c1f09a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 >> 172.21.15.181:0/1032821232 conn(0x7f5d2c06b580 msgr2=0x7f5d2c158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 shutdown_connections 2024-03-29T04:09:40.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.248+0000 7f5d3284a640 1 -- 172.21.15.181:0/1032821232 wait complete. 2024-03-29T04:09:40.239 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:40.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.361+0000 7fd45b8d3640 1 Processor -- start 2024-03-29T04:09:40.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.361+0000 7fd45b8d3640 1 -- start start 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd45b8d3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4540cf070 0x7fd4540cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd45b8d3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd4540cbbd0 0x7fd45415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd45b8d3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4540cbfe0 con 0x7fd4541445e0 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd45b8d3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd4540cd610 con 0x7fd4540cf070 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd45b8d3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd4541449c0 con 0x7fd4540cbbd0 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4540cf070 0x7fd4540cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4540cf070 0x7fd4540cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32898/0 (socket says 172.21.15.181:32898) 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 -- 172.21.15.181:0/1748288710 learned_addr learned my addr 172.21.15.181:0/1748288710 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd459e70640 1 --2- 172.21.15.181:0/1748288710 >> v2:172.21.15.187:3300/0 conn(0x7fd4540cbbd0 0x7fd45415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 -- 172.21.15.181:0/1748288710 >> v2:172.21.15.187:3300/0 conn(0x7fd4540cbbd0 msgr2=0x7fd45415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 --2- 172.21.15.181:0/1748288710 >> v2:172.21.15.187:3300/0 conn(0x7fd4540cbbd0 0x7fd45415f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 -- 172.21.15.181:0/1748288710 >> v1:172.21.15.187:6789/0 conn(0x7fd4541445e0 legacy=0x7fd454072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.362+0000 7fd458e6e640 1 -- 172.21.15.181:0/1748288710 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd454144e10 con 0x7fd4540cf070 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd458e6e640 1 --2- 172.21.15.181:0/1748288710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540cb690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd45406f610 tx=0x7fd44402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e520eb62141c023 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd44bfff640 1 -- 172.21.15.181:0/1748288710 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444034070 con 0x7fd4540cf070 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd44bfff640 1 -- 172.21.15.181:0/1748288710 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd444030860 con 0x7fd4540cf070 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd44bfff640 1 -- 172.21.15.181:0/1748288710 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444030c30 con 0x7fd4540cf070 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 -- 172.21.15.181:0/1748288710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 msgr2=0x7fd4540cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/1748288710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540cb690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd45406f610 tx=0x7fd44402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 -- 172.21.15.181:0/1748288710 shutdown_connections 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/1748288710 >> v2:172.21.15.187:3300/0 conn(0x7fd4540cbbd0 0x7fd45415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/1748288710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540cb690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.363+0000 7fd45b8d3640 1 -- 172.21.15.181:0/1748288710 >> 172.21.15.181:0/1748288710 conn(0x7fd4540b1c90 msgr2=0x7fd4540b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.364+0000 7fd45b8d3640 1 -- 172.21.15.181:0/1748288710 shutdown_connections 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.364+0000 7fd45b8d3640 1 -- 172.21.15.181:0/1748288710 wait complete. 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 Processor -- start 2024-03-29T04:09:40.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 -- start start 2024-03-29T04:09:40.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4540cbbd0 0x7fd45415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4541445e0 0x7fd4540b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4540cad60 con 0x7fd4540cf070 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd45415edb0 con 0x7fd4541445e0 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45b8d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4540cb040 con 0x7fd4540cbbd0 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd458e6e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd458e6e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32910/0 (socket says 172.21.15.181:32910) 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd458e6e640 1 -- 172.21.15.181:0/973745943 learned_addr learned my addr 172.21.15.181:0/973745943 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd459e70640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4541445e0 0x7fd4540b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.365+0000 7fd45966f640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4540cbbd0 0x7fd45415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 -- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4540cbbd0 msgr2=0x7fd45415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4540cbbd0 0x7fd45415d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 -- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4541445e0 msgr2=0x7fd4540b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4541445e0 0x7fd4540b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 -- 172.21.15.181:0/973745943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4540b5790 con 0x7fd4540cf070 2024-03-29T04:09:40.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd458e6e640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd44402b9e0 tx=0x7fd444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444048070 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd444030660 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4540b5a70 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444034040 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.366+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4540b3250 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.367+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd454075a10 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.368+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd44404b070 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.368+0000 7fd449ffb640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 0x7fd43806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.368+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd4440ad980 con 0x7fd4540cf070 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.368+0000 7fd45966f640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 0x7fd43806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.369+0000 7fd45966f640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 0x7fd43806e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4541312a0 tx=0x7fd450009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.373+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd44407b610 con 0x7fd4540cf070 2024-03-29T04:09:40.526 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.547+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4540cd790 con 0x7fd4540cf070 2024-03-29T04:09:40.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.548+0000 7fd449ffb640 1 -- 172.21.15.181:0/973745943 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fd44407f910 con 0x7fd4540cf070 2024-03-29T04:09:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:40.528 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 msgr2=0x7fd43806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 0x7fd43806e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4541312a0 tx=0x7fd450009040 comp rx=0 tx=0).stop 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 msgr2=0x7fd4540b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd44402b9e0 tx=0x7fd444004730 comp rx=0 tx=0).stop 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 shutdown_connections 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4541445e0 0x7fd4540b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4540cf070 0x7fd4540b2d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd43806c530 0x7fd43806e9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 --2- 172.21.15.181:0/973745943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4540cbbd0 0x7fd45415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 >> 172.21.15.181:0/973745943 conn(0x7fd4540b1c90 msgr2=0x7fd4540d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 shutdown_connections 2024-03-29T04:09:40.531 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.550+0000 7fd45b8d3640 1 -- 172.21.15.181:0/973745943 wait complete. 2024-03-29T04:09:40.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:40.542 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.664+0000 7fe4d2d74640 1 Processor -- start 2024-03-29T04:09:40.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.664+0000 7fe4d2d74640 1 -- start start 2024-03-29T04:09:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d2d74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0b5dc0 0x7fe4cc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d2d74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4cc0b73d0 0x7fe4cc0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d2d74640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc0b66e0 con 0x7fe4cc0ba090 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d2d74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe4cc0d2d90 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d2d74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc0d2ff0 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d1311640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4cc0b73d0 0x7fe4cc0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4cbfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe4cc0ba090 0x7fe4cc0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34722/0 (socket says 172.21.15.181:34722) 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4cbfff640 1 -- 172.21.15.181:0/197934588 learned_addr learned my addr 172.21.15.181:0/197934588 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.665+0000 7fe4d0b10640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0b5dc0 0x7fe4cc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4d1311640 1 -- 172.21.15.181:0/197934588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0b5dc0 msgr2=0x7fe4cc0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4d1311640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0b5dc0 0x7fe4cc0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4d1311640 1 -- 172.21.15.181:0/197934588 >> v1:172.21.15.181:6789/0 conn(0x7fe4cc0ba090 legacy=0x7fe4cc0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4d1311640 1 -- 172.21.15.181:0/197934588 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc0d32d0 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4cb7fe640 1 -- 172.21.15.181:0/197934588 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296821294 0 0) 0x7fe4cc0b66e0 con 0x7fe4cc0ba090 2024-03-29T04:09:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.666+0000 7fe4d1311640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc0b7880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009d90 tx=0x7fe4c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2769d2c78b73afe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4cb7fe640 1 -- 172.21.15.181:0/197934588 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4039070 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4cb7fe640 1 -- 172.21.15.181:0/197934588 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c402cc50 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4cb7fe640 1 -- 172.21.15.181:0/197934588 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4034440 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 -- 172.21.15.181:0/197934588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b73d0 msgr2=0x7fe4cc0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc0b7880 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009d90 tx=0x7fe4c402bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 -- 172.21.15.181:0/197934588 shutdown_connections 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc0b7880 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/197934588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0b5dc0 0x7fe4cc0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 -- 172.21.15.181:0/197934588 >> 172.21.15.181:0/197934588 conn(0x7fe4cc0678f0 msgr2=0x7fe4cc0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 -- 172.21.15.181:0/197934588 shutdown_connections 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.667+0000 7fe4d2d74640 1 -- 172.21.15.181:0/197934588 wait complete. 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.668+0000 7fe4d2d74640 1 Processor -- start 2024-03-29T04:09:40.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.668+0000 7fe4d2d74640 1 -- start start 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc1613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 0x7fe4cc15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0d3b70 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc0d3890 con 0x7fe4cc0b73d0 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d2d74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc0d4090 con 0x7fe4cc0ba090 2024-03-29T04:09:40.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d0b10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d0b10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32918/0 (socket says 172.21.15.181:32918) 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d0b10640 1 -- 172.21.15.181:0/3609893858 learned_addr learned my addr 172.21.15.181:0/3609893858 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4d1311640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 0x7fe4cc15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.669+0000 7fe4cbfff640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc1613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 -- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 msgr2=0x7fe4cc15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 0x7fe4cc15b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 -- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4cc0b73d0 msgr2=0x7fe4cc1613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc1613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 -- 172.21.15.181:0/3609893858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc14eee0 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d1311640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 0x7fe4cc15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d0b10640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e930 tx=0x7fe4bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc017070 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00ca00 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc160350 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00cdf0 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.670+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc160910 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.671+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc075890 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.671+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe4bc01a070 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.672+0000 7fe4c97fa640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 0x7fe4b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.672+0000 7fe4cbfff640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 0x7fe4b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.672+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe4bc08b910 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.672+0000 7fe4cbfff640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 0x7fe4b806e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0b59a0 tx=0x7fe4c0007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.676+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4cc14eee0 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.848+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4cc15d340 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.848+0000 7fe4c97fa640 1 -- 172.21.15.181:0/3609893858 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe4bc0596f0 con 0x7fe4cc0b5dc0 2024-03-29T04:09:40.828 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 msgr2=0x7fe4b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 0x7fe4b806e9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0b59a0 tx=0x7fe4c0007de0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 msgr2=0x7fe4cc14e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e930 tx=0x7fe4bc00edf0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 shutdown_connections 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe4b806c530 0x7fe4b806e9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4cc0ba090 0x7fe4cc15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4cc0b73d0 0x7fe4cc1613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 --2- 172.21.15.181:0/3609893858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4cc0b5dc0 0x7fe4cc14e9a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.830 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 >> 172.21.15.181:0/3609893858 conn(0x7fe4cc0678f0 msgr2=0x7fe4cc0b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 shutdown_connections 2024-03-29T04:09:40.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.850+0000 7fe4d2d74640 1 -- 172.21.15.181:0/3609893858 wait complete. 2024-03-29T04:09:40.841 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:40.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.964+0000 7f516de4a640 1 Processor -- start 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.965+0000 7f516de4a640 1 -- start start 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.965+0000 7f516de4a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51680b5dc0 0x7f51680b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.965+0000 7f516de4a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.965+0000 7f516de4a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51680b66e0 con 0x7f51680ba090 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.965+0000 7f516de4a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f51680d2d90 con 0x7f51680b73d0 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f516de4a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f51680d2ff0 con 0x7f51680b5dc0 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f5167fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51680b5dc0 0x7f51680b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f5167fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32930/0 (socket says 172.21.15.181:32930) 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51680b5dc0 0x7f51680b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:58054/0 (socket says 172.21.15.181:58054) 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f5167fff640 1 -- 172.21.15.181:0/3873610256 learned_addr learned my addr 172.21.15.181:0/3873610256 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.948 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f5166ffd640 1 --1- 172.21.15.181:0/3873610256 >> v1:172.21.15.181:6789/0 conn(0x7f51680ba090 0x7f51680d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34726/0 (socket says 172.21.15.181:34726) 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 -- 172.21.15.181:0/3873610256 >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 msgr2=0x7f51680b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 --2- 172.21.15.181:0/3873610256 >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 -- 172.21.15.181:0/3873610256 >> v1:172.21.15.181:6789/0 conn(0x7f51680ba090 legacy=0x7f51680d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f51677fe640 1 -- 172.21.15.181:0/3873610256 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51680d32d0 con 0x7f51680b5dc0 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.966+0000 7f5167fff640 1 --2- 172.21.15.181:0/3873610256 >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f51677fe640 1 --2- 172.21.15.181:0/3873610256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b5dc0 0x7f51680b61a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f516806d770 tx=0x7f515802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d8b665bffdb43b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f51667fc640 1 -- 172.21.15.181:0/3873610256 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158035020 con 0x7f51680b5dc0 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f51667fc640 1 -- 172.21.15.181:0/3873610256 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5158031830 con 0x7f51680b5dc0 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f51667fc640 1 -- 172.21.15.181:0/3873610256 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158031c00 con 0x7f51680b5dc0 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 -- 172.21.15.181:0/3873610256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b5dc0 msgr2=0x7f51680b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 --2- 172.21.15.181:0/3873610256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b5dc0 0x7f51680b61a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f516806d770 tx=0x7f515802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 -- 172.21.15.181:0/3873610256 shutdown_connections 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 --2- 172.21.15.181:0/3873610256 >> v2:172.21.15.181:3300/0 conn(0x7f51680b73d0 0x7f51680b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 --2- 172.21.15.181:0/3873610256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b5dc0 0x7f51680b61a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.949 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 -- 172.21.15.181:0/3873610256 >> 172.21.15.181:0/3873610256 conn(0x7f51680678f0 msgr2=0x7f51680d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 -- 172.21.15.181:0/3873610256 shutdown_connections 2024-03-29T04:09:40.950 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.967+0000 7f516de4a640 1 -- 172.21.15.181:0/3873610256 wait complete. 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 Processor -- start 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 -- start start 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 0x7f51680cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 0x7f51680cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51680d3b70 con 0x7f51680b5dc0 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51680d3890 con 0x7f51680b73d0 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.969+0000 7f516de4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51680d4090 con 0x7f51680ba090 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32936/0 (socket says 172.21.15.181:32936) 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 -- 172.21.15.181:0/592811094 learned_addr learned my addr 172.21.15.181:0/592811094 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:40.953 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f5166ffd640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 0x7f51680cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f5167fff640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 0x7f51680cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 -- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 msgr2=0x7f51680cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 0x7f51680cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 -- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 msgr2=0x7f51680cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 0x7f51680cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 -- 172.21.15.181:0/592811094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516815fa70 con 0x7f51680b5dc0 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f5166ffd640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 0x7f51680cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f5167fff640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 0x7f51680cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f51677fe640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5158009890 tx=0x7f51580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158041070 con 0x7f51680b5dc0 2024-03-29T04:09:40.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515803b920 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51680cde10 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158035040 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.970+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516815fd10 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.971+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5168075890 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.972+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5158038020 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.972+0000 7f514ffff640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 0x7f515406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.972+0000 7f5166ffd640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 0x7f515406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.972+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f51580ad3b0 con 0x7f51680b5dc0 2024-03-29T04:09:40.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.972+0000 7f5166ffd640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 0x7f515406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f515c007fd0 tx=0x7f515c007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:40.956 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:40.976+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f515802f2e0 con 0x7f51680b5dc0 2024-03-29T04:09:41.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.150+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f516806b580 con 0x7f51680b5dc0 2024-03-29T04:09:41.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.151+0000 7f514ffff640 1 -- 172.21.15.181:0/592811094 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f5158031800 con 0x7f51680b5dc0 2024-03-29T04:09:41.130 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:41.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 msgr2=0x7f515406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 0x7f515406e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f515c007fd0 tx=0x7f515c007d60 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 msgr2=0x7f51681e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5158009890 tx=0x7f51580046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 shutdown_connections 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f515406c530 0x7f515406e9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51680ba090 0x7f51680cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51680b73d0 0x7f51680cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 --2- 172.21.15.181:0/592811094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51680b5dc0 0x7f51681e7710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 >> 172.21.15.181:0/592811094 conn(0x7f51680678f0 msgr2=0x7f51680d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 shutdown_connections 2024-03-29T04:09:41.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.153+0000 7f516de4a640 1 -- 172.21.15.181:0/592811094 wait complete. 2024-03-29T04:09:41.143 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:41.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.267+0000 7f2636a2a640 1 Processor -- start 2024-03-29T04:09:41.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.267+0000 7f2636a2a640 1 -- start start 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2636a2a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26300b2f20 0x7f263015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2636a2a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26300b35e0 con 0x7f2630144cb0 2024-03-29T04:09:41.248 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2636a2a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26300b38c0 con 0x7f2630072d50 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2636a2a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f26300b3e80 con 0x7f26300b2f20 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2634fc7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26300b2f20 0x7f263015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f262ffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2630144cb0 0x7f2630072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34740/0 (socket says 172.21.15.181:34740) 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f262ffff640 1 -- 172.21.15.181:0/3811780796 learned_addr learned my addr 172.21.15.181:0/3811780796 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2634fc7640 1 -- 172.21.15.181:0/3811780796 >> v1:172.21.15.187:6789/0 conn(0x7f2630072d50 legacy=0x7f26300ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2634fc7640 1 -- 172.21.15.181:0/3811780796 >> v1:172.21.15.181:6789/0 conn(0x7f2630144cb0 legacy=0x7f2630072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f262effd640 1 -- 172.21.15.181:0/3811780796 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685700447 0 0) 0x7f26300b38c0 con 0x7f2630072d50 2024-03-29T04:09:41.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.268+0000 7f2634fc7640 1 -- 172.21.15.181:0/3811780796 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26300b3ba0 con 0x7f26300b2f20 2024-03-29T04:09:41.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.269+0000 7f2634fc7640 1 --2- 172.21.15.181:0/3811780796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26300b2f20 0x7f263015f800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d90 tx=0x7f262802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b520973208e9ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.269+0000 7f262effd640 1 -- 172.21.15.181:0/3811780796 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628039070 con 0x7f26300b2f20 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.269+0000 7f262effd640 1 -- 172.21.15.181:0/3811780796 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262802cab0 con 0x7f26300b2f20 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.269+0000 7f262effd640 1 -- 172.21.15.181:0/3811780796 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628034440 con 0x7f26300b2f20 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 -- 172.21.15.181:0/3811780796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26300b2f20 msgr2=0x7f263015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 --2- 172.21.15.181:0/3811780796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26300b2f20 0x7f263015f800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d90 tx=0x7f262802bd80 comp rx=0 tx=0).stop 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 -- 172.21.15.181:0/3811780796 shutdown_connections 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 --2- 172.21.15.181:0/3811780796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26300b2f20 0x7f263015f800 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 -- 172.21.15.181:0/3811780796 >> 172.21.15.181:0/3811780796 conn(0x7f26300b5b00 msgr2=0x7f263006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 -- 172.21.15.181:0/3811780796 shutdown_connections 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.270+0000 7f2636a2a640 1 -- 172.21.15.181:0/3811780796 wait complete. 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.271+0000 7f2636a2a640 1 Processor -- start 2024-03-29T04:09:41.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.271+0000 7f2636a2a640 1 -- start start 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2630072d50 0x7f263013f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26300b2f20 0x7f2630154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26300ce760 con 0x7f2630144cb0 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26300b3390 con 0x7f26300b2f20 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2636a2a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26300ce8e0 con 0x7f2630072d50 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32954/0 (socket says 172.21.15.181:32954) 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 -- 172.21.15.181:0/674676687 learned_addr learned my addr 172.21.15.181:0/674676687 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:41.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f262f7fe640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26300b2f20 0x7f2630154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f262ffff640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2630072d50 0x7f263013f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 -- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2630072d50 msgr2=0x7f263013f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2630072d50 0x7f263013f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 -- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26300b2f20 msgr2=0x7f2630154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26300b2f20 0x7f2630154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 -- 172.21.15.181:0/674676687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630140490 con 0x7f2630144cb0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f2634fc7640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2628037040 tx=0x7f2628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.272+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628044020 con 0x7f2630144cb0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.273+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262803a040 con 0x7f2630144cb0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.273+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628039050 con 0x7f2630144cb0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.273+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263013af20 con 0x7f2630144cb0 2024-03-29T04:09:41.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.273+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2630140a10 con 0x7f2630144cb0 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.274+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2630075890 con 0x7f2630144cb0 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.274+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f262802c970 con 0x7f2630144cb0 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.274+0000 7f262cff9640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 0x7f261c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.274+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f26280adf30 con 0x7f2630144cb0 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.275+0000 7f262ffff640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 0x7f261c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.275+0000 7f262ffff640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 0x7f261c06e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2620000f60 tx=0x7f2620007e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.279+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f262804e060 con 0x7f2630144cb0 2024-03-29T04:09:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.453+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26300b4030 con 0x7f2630144cb0 2024-03-29T04:09:41.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.454+0000 7f262cff9640 1 -- 172.21.15.181:0/674676687 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f262807bc50 con 0x7f2630144cb0 2024-03-29T04:09:41.433 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:41.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 msgr2=0x7f261c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 0x7f261c06e9f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2620000f60 tx=0x7f2620007e20 comp rx=0 tx=0).stop 2024-03-29T04:09:41.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 msgr2=0x7f2630155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2628037040 tx=0x7f2628004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 shutdown_connections 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2630144cb0 0x7f2630155320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26300b2f20 0x7f2630154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f261c06c530 0x7f261c06e9f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 --2- 172.21.15.181:0/674676687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2630072d50 0x7f263013f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.455+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 >> 172.21.15.181:0/674676687 conn(0x7f26300b5b00 msgr2=0x7f26300714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.456+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 shutdown_connections 2024-03-29T04:09:41.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.456+0000 7f2636a2a640 1 -- 172.21.15.181:0/674676687 wait complete. 2024-03-29T04:09:41.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:41.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.570+0000 7f612cd9d640 1 Processor -- start 2024-03-29T04:09:41.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.570+0000 7f612cd9d640 1 -- start start 2024-03-29T04:09:41.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f612cd9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61280b4920 0x7f61280b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f612cd9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61280b4d00 0x7f61280d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f612cd9d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280b54b0 con 0x7f61280d0b90 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f612cd9d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f61280d0260 con 0x7f61280b4920 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f612cd9d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f61280cfca0 con 0x7f61280b4d00 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61280b4920 0x7f61280b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61280b4920 0x7f61280b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32970/0 (socket says 172.21.15.181:32970) 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 -- 172.21.15.181:0/471372524 learned_addr learned my addr 172.21.15.181:0/471372524 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f6126ffd640 1 --2- 172.21.15.181:0/471372524 >> v2:172.21.15.187:3300/0 conn(0x7f61280b4d00 0x7f61280d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 -- 172.21.15.181:0/471372524 >> v2:172.21.15.187:3300/0 conn(0x7f61280b4d00 msgr2=0x7f61280d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 --2- 172.21.15.181:0/471372524 >> v2:172.21.15.187:3300/0 conn(0x7f61280b4d00 0x7f61280d4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 -- 172.21.15.181:0/471372524 >> v1:172.21.15.187:6789/0 conn(0x7f61280d0b90 legacy=0x7f61280d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.571+0000 7f61277fe640 1 -- 172.21.15.181:0/471372524 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280cff80 con 0x7f61280b4920 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f61277fe640 1 --2- 172.21.15.181:0/471372524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4920 0x7f61280b3ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61281621f0 tx=0x7f611802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dee2dcbe87133c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f61267fc640 1 -- 172.21.15.181:0/471372524 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118035020 con 0x7f61280b4920 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f61267fc640 1 -- 172.21.15.181:0/471372524 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118031870 con 0x7f61280b4920 2024-03-29T04:09:41.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f61267fc640 1 -- 172.21.15.181:0/471372524 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118031c40 con 0x7f61280b4920 2024-03-29T04:09:41.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f612cd9d640 1 -- 172.21.15.181:0/471372524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4920 msgr2=0x7f61280b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.572+0000 7f612cd9d640 1 --2- 172.21.15.181:0/471372524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4920 0x7f61280b3ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61281621f0 tx=0x7f611802b910 comp rx=0 tx=0).stop 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 -- 172.21.15.181:0/471372524 shutdown_connections 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 --2- 172.21.15.181:0/471372524 >> v2:172.21.15.187:3300/0 conn(0x7f61280b4d00 0x7f61280d4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 --2- 172.21.15.181:0/471372524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4920 0x7f61280b3ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 -- 172.21.15.181:0/471372524 >> 172.21.15.181:0/471372524 conn(0x7f612809d4e0 msgr2=0x7f61280d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 -- 172.21.15.181:0/471372524 shutdown_connections 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.573+0000 7f612cd9d640 1 -- 172.21.15.181:0/471372524 wait complete. 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.574+0000 7f612cd9d640 1 Processor -- start 2024-03-29T04:09:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.574+0000 7f612cd9d640 1 -- start start 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61280b4920 0x7f6128152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280d0b90 0x7f612806d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61280b3ff0 con 0x7f61280b4d00 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61280d2180 con 0x7f61280d0b90 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61280d5000 con 0x7f61280b4920 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32982/0 (socket says 172.21.15.181:32982) 2024-03-29T04:09:41.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 -- 172.21.15.181:0/2346940692 learned_addr learned my addr 172.21.15.181:0/2346940692 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f61277fe640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61280b4920 0x7f6128152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6127fff640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280d0b90 0x7f612806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 -- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61280b4920 msgr2=0x7f6128152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61280b4920 0x7f6128152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 -- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280d0b90 msgr2=0x7f612806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280d0b90 0x7f612806d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 -- 172.21.15.181:0/2346940692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612809ef90 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f6126ffd640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9b0 tx=0x7f611c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c017070 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c00c9a0 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612809f270 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00cd90 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.575+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612809f830 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.576+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6128075890 con 0x7f61280b4d00 2024-03-29T04:09:41.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.577+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f611c01a070 con 0x7f61280b4d00 2024-03-29T04:09:41.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.577+0000 7f610ffff640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 0x7f611406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.577+0000 7f61277fe640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 0x7f611406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.577+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f611c08b8c0 con 0x7f61280b4d00 2024-03-29T04:09:41.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.578+0000 7f61277fe640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 0x7f611406e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6118036000 tx=0x7f611803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.581+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f611c0595e0 con 0x7f61280b4d00 2024-03-29T04:09:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.756+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f61280b3510 con 0x7f61280b4d00 2024-03-29T04:09:41.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.757+0000 7f610ffff640 1 -- 172.21.15.181:0/2346940692 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f611c05d8e0 con 0x7f61280b4d00 2024-03-29T04:09:41.737 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 msgr2=0x7f611406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 0x7f611406e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6118036000 tx=0x7f611803c040 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 msgr2=0x7f612806d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f611c00e9b0 tx=0x7f611c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 shutdown_connections 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61280d0b90 0x7f612806d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61280b4d00 0x7f612806d2e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f611406c530 0x7f611406e9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 --2- 172.21.15.181:0/2346940692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61280b4920 0x7f6128152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 >> 172.21.15.181:0/2346940692 conn(0x7f612809d4e0 msgr2=0x7f61280b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 shutdown_connections 2024-03-29T04:09:41.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.759+0000 7f612cd9d640 1 -- 172.21.15.181:0/2346940692 wait complete. 2024-03-29T04:09:41.750 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 Processor -- start 2024-03-29T04:09:41.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 -- start start 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99a815e950 0x7f99a80d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d30d0 0x7f99a80b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a80d36f0 con 0x7f99a80d48c0 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f99a80d3cb0 con 0x7f99a815e950 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99aee74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99a80d4d00 con 0x7f99a80d30d0 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99acc10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99a815e950 0x7f99a80d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99acc10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99a815e950 0x7f99a80d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32992/0 (socket says 172.21.15.181:32992) 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.871+0000 7f99acc10640 1 -- 172.21.15.181:0/1894071777 learned_addr learned my addr 172.21.15.181:0/1894071777 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:41.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99ad411640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d30d0 0x7f99a80b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99acc10640 1 -- 172.21.15.181:0/1894071777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d30d0 msgr2=0x7f99a80b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99acc10640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d30d0 0x7f99a80b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99acc10640 1 -- 172.21.15.181:0/1894071777 >> v1:172.21.15.187:6789/0 conn(0x7f99a80d48c0 legacy=0x7f99a80d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99acc10640 1 -- 172.21.15.181:0/1894071777 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80d39d0 con 0x7f99a815e950 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.872+0000 7f99acc10640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a815e950 0x7f99a80d4380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f999c009870 tx=0x7f999c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46be0a3332285ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99a77fe640 1 -- 172.21.15.181:0/1894071777 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c02f840 con 0x7f99a815e950 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99a77fe640 1 -- 172.21.15.181:0/1894071777 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c02f9e0 con 0x7f99a815e950 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99a77fe640 1 -- 172.21.15.181:0/1894071777 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c02fdb0 con 0x7f99a815e950 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 -- 172.21.15.181:0/1894071777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a815e950 msgr2=0x7f99a80d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a815e950 0x7f99a80d4380 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f999c009870 tx=0x7f999c02ba20 comp rx=0 tx=0).stop 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 -- 172.21.15.181:0/1894071777 shutdown_connections 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d30d0 0x7f99a80b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 --2- 172.21.15.181:0/1894071777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a815e950 0x7f99a80d4380 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 -- 172.21.15.181:0/1894071777 >> 172.21.15.181:0/1894071777 conn(0x7f99a80676e0 msgr2=0x7f99a8067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:41.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 -- 172.21.15.181:0/1894071777 shutdown_connections 2024-03-29T04:09:41.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.873+0000 7f99aee74640 1 -- 172.21.15.181:0/1894071777 wait complete. 2024-03-29T04:09:41.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.874+0000 7f99aee74640 1 Processor -- start 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.874+0000 7f99aee74640 1 -- start start 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 0x7f99a8156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a815e950 0x7f99a8158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f99a815ed30 con 0x7f99a80d30d0 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80b7da0 con 0x7f99a815e950 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99aee74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f99a815eeb0 con 0x7f99a80d48c0 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99acc10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99acc10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:32994/0 (socket says 172.21.15.181:32994) 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99acc10640 1 -- 172.21.15.181:0/1641860568 learned_addr learned my addr 172.21.15.181:0/1641860568 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99a7fff640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 0x7f99a8156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.875+0000 7f99ad411640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a815e950 0x7f99a8158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 -- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 msgr2=0x7f99a8156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 0x7f99a8156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 -- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a815e950 msgr2=0x7f99a8158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a815e950 0x7f99a8158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 -- 172.21.15.181:0/1641860568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a8159360 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99a7fff640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 0x7f99a8156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99acc10640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c037bc0 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c046070 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a8159640 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c041400 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.876+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a80ca2c0 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.877+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8075a10 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.877+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f999c040070 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.878+0000 7f99a57fa640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 0x7f998c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.878+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f999c0adf10 con 0x7f99a80d30d0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.878+0000 7f99a7fff640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 0x7f998c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:41.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.878+0000 7f99a7fff640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 0x7f998c06e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99a815fe10 tx=0x7f9994009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:41.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:41.882+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f999c033030 con 0x7f99a80d30d0 2024-03-29T04:09:42.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.093+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a80cc870 con 0x7f99a80d30d0 2024-03-29T04:09:42.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.094+0000 7f99a57fa640 1 -- 172.21.15.181:0/1641860568 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f999c07bb10 con 0x7f99a80d30d0 2024-03-29T04:09:42.074 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:42.074 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:42.074 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:09:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.095+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 msgr2=0x7f998c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 0x7f998c06e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f99a815fe10 tx=0x7f9994009040 comp rx=0 tx=0).stop 2024-03-29T04:09:42.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 msgr2=0x7f99a8155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c004730 comp rx=0 tx=0).stop 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 shutdown_connections 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a815e950 0x7f99a8158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f998c06c530 0x7f998c06e9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f99a80d48c0 0x7f99a8156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 --2- 172.21.15.181:0/1641860568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a80d30d0 0x7f99a8155020 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 >> 172.21.15.181:0/1641860568 conn(0x7f99a80676e0 msgr2=0x7f99a80d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 shutdown_connections 2024-03-29T04:09:42.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.096+0000 7f99aee74640 1 -- 172.21.15.181:0/1641860568 wait complete. 2024-03-29T04:09:42.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:42.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 Processor -- start 2024-03-29T04:09:42.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 -- start start 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd38c15e950 0x7fd38c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd38c0d30d0 0x7fd38c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd38c0d36f0 con 0x7fd38c0d48c0 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd38c0d3cb0 con 0x7fd38c0d30d0 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.209+0000 7fd39323c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd38c0d4d00 con 0x7fd38c15e950 2024-03-29T04:09:42.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3917d9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd38c0d30d0 0x7fd38c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd383fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd38c0d48c0 0x7fd38c0d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34754/0 (socket says 172.21.15.181:34754) 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd383fff640 1 -- 172.21.15.181:0/1456064250 learned_addr learned my addr 172.21.15.181:0/1456064250 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd390fd8640 1 --2- 172.21.15.181:0/1456064250 >> v2:172.21.15.187:3300/0 conn(0x7fd38c15e950 0x7fd38c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3917d9640 1 -- 172.21.15.181:0/1456064250 >> v2:172.21.15.187:3300/0 conn(0x7fd38c15e950 msgr2=0x7fd38c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3917d9640 1 --2- 172.21.15.181:0/1456064250 >> v2:172.21.15.187:3300/0 conn(0x7fd38c15e950 0x7fd38c0d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3917d9640 1 -- 172.21.15.181:0/1456064250 >> v1:172.21.15.181:6789/0 conn(0x7fd38c0d48c0 legacy=0x7fd38c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3917d9640 1 -- 172.21.15.181:0/1456064250 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c0d39d0 con 0x7fd38c0d30d0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.210+0000 7fd3837fe640 1 -- 172.21.15.181:0/1456064250 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299146207 0 0) 0x7fd38c0d36f0 con 0x7fd38c0d48c0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd3917d9640 1 --2- 172.21.15.181:0/1456064250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d30d0 0x7fd38c0b82c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0d3e30 tx=0x7fd38402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86735b01e515f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd3837fe640 1 -- 172.21.15.181:0/1456064250 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384039070 con 0x7fd38c0d30d0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd3837fe640 1 -- 172.21.15.181:0/1456064250 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38402cd00 con 0x7fd38c0d30d0 2024-03-29T04:09:42.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd3837fe640 1 -- 172.21.15.181:0/1456064250 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384034660 con 0x7fd38c0d30d0 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 -- 172.21.15.181:0/1456064250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d30d0 msgr2=0x7fd38c0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 --2- 172.21.15.181:0/1456064250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d30d0 0x7fd38c0b82c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0d3e30 tx=0x7fd38402bf50 comp rx=0 tx=0).stop 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 -- 172.21.15.181:0/1456064250 shutdown_connections 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 --2- 172.21.15.181:0/1456064250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d30d0 0x7fd38c0b82c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 --2- 172.21.15.181:0/1456064250 >> v2:172.21.15.187:3300/0 conn(0x7fd38c15e950 0x7fd38c0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 -- 172.21.15.181:0/1456064250 >> 172.21.15.181:0/1456064250 conn(0x7fd38c0676e0 msgr2=0x7fd38c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 -- 172.21.15.181:0/1456064250 shutdown_connections 2024-03-29T04:09:42.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.211+0000 7fd39323c640 1 -- 172.21.15.181:0/1456064250 wait complete. 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.212+0000 7fd39323c640 1 Processor -- start 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 -- start start 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd38c0d30d0 0x7fd38c1e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd38c15e950 0x7fd38c1e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c15ed30 con 0x7fd38c0d48c0 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c0b7da0 con 0x7fd38c15e950 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd39323c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c15eeb0 con 0x7fd38c0d30d0 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd383fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd383fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33020/0 (socket says 172.21.15.181:33020) 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd383fff640 1 -- 172.21.15.181:0/4203638071 learned_addr learned my addr 172.21.15.181:0/4203638071 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd390fd8640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd38c0d30d0 0x7fd38c1e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.213+0000 7fd3917d9640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd38c15e950 0x7fd38c1e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 -- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd38c0d30d0 msgr2=0x7fd38c1e8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd38c0d30d0 0x7fd38c1e8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 -- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd38c15e950 msgr2=0x7fd38c1e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd38c15e950 0x7fd38c1e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 -- 172.21.15.181:0/4203638071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c0c9940 con 0x7fd38c0d48c0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd383fff640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c4d0 tx=0x7fd37c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c018070 con 0x7fd38c0d48c0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c007500 con 0x7fd38c0d48c0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c0c9c20 con 0x7fd38c0d48c0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c004a20 con 0x7fd38c0d48c0 2024-03-29T04:09:42.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.214+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c0ca1e0 con 0x7fd38c0d48c0 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.215+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0d1480 con 0x7fd38c0d48c0 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.216+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd37c01b070 con 0x7fd38c0d48c0 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.216+0000 7fd3817fa640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 0x7fd37806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.216+0000 7fd390fd8640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 0x7fd37806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.216+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd37c08ab90 con 0x7fd38c0d48c0 2024-03-29T04:09:42.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.216+0000 7fd390fd8640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 0x7fd37806ea20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0c9030 tx=0x7fd388009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.220+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd38c0d1480 con 0x7fd38c0d48c0 2024-03-29T04:09:42.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.398+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd38c075a10 con 0x7fd38c0d48c0 2024-03-29T04:09:42.378 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.399+0000 7fd3817fa640 1 -- 172.21.15.181:0/4203638071 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fd37c0588b0 con 0x7fd38c0d48c0 2024-03-29T04:09:42.379 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:42.379 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 msgr2=0x7fd37806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 0x7fd37806ea20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0c9030 tx=0x7fd388009190 comp rx=0 tx=0).stop 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 msgr2=0x7fd38c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00c4d0 tx=0x7fd37c00c990 comp rx=0 tx=0).stop 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 shutdown_connections 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd38c15e950 0x7fd38c1e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c0d48c0 0x7fd38c0ca700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd37806c560 0x7fd37806ea20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 --2- 172.21.15.181:0/4203638071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd38c0d30d0 0x7fd38c1e8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 >> 172.21.15.181:0/4203638071 conn(0x7fd38c0676e0 msgr2=0x7fd38c0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 shutdown_connections 2024-03-29T04:09:42.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.401+0000 7fd39323c640 1 -- 172.21.15.181:0/4203638071 wait complete. 2024-03-29T04:09:42.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:09:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.512+0000 7f41a6dd4640 1 Processor -- start 2024-03-29T04:09:42.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 -- start start 2024-03-29T04:09:42.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a00d36f0 con 0x7f41a00d48c0 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f41a00d3cb0 con 0x7f41a015e950 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a6dd4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00d4d00 con 0x7f41a00d30d0 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a4b70640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f41a5371640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.513+0000 7f419ffff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f41a00d48c0 0x7f41a00d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:60662/0 (socket says 172.21.15.181:60662) 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a5371640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:45390/0 (socket says 172.21.15.181:45390) 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f419ffff640 1 -- 172.21.15.181:0/3407198005 learned_addr learned my addr 172.21.15.181:0/3407198005 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a4b70640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:58122/0 (socket says 172.21.15.181:58122) 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a5371640 1 -- 172.21.15.181:0/3407198005 >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 msgr2=0x7f41a00d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a5371640 1 --2- 172.21.15.181:0/3407198005 >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a5371640 1 -- 172.21.15.181:0/3407198005 >> v1:172.21.15.187:6789/0 conn(0x7f41a00d48c0 legacy=0x7f41a00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a5371640 1 -- 172.21.15.181:0/3407198005 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00d39d0 con 0x7f41a00d30d0 2024-03-29T04:09:42.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.514+0000 7f41a4b70640 1 --2- 172.21.15.181:0/3407198005 >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a5371640 1 --2- 172.21.15.181:0/3407198005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4198009d90 tx=0x7f419802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80878f0b05644990 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f419f7fe640 1 -- 172.21.15.181:0/3407198005 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198039070 con 0x7f41a00d30d0 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f419f7fe640 1 -- 172.21.15.181:0/3407198005 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f419802cce0 con 0x7f41a00d30d0 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f419f7fe640 1 -- 172.21.15.181:0/3407198005 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198034660 con 0x7f41a00d30d0 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 -- 172.21.15.181:0/3407198005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 msgr2=0x7f41a00b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/3407198005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4198009d90 tx=0x7f419802bf30 comp rx=0 tx=0).stop 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 -- 172.21.15.181:0/3407198005 shutdown_connections 2024-03-29T04:09:42.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/3407198005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d30d0 0x7f41a00b82c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/3407198005 >> v2:172.21.15.187:3300/0 conn(0x7f41a015e950 0x7f41a00d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 -- 172.21.15.181:0/3407198005 >> 172.21.15.181:0/3407198005 conn(0x7f41a00676e0 msgr2=0x7f41a0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 -- 172.21.15.181:0/3407198005 shutdown_connections 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.515+0000 7f41a6dd4640 1 -- 172.21.15.181:0/3407198005 wait complete. 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.516+0000 7f41a6dd4640 1 Processor -- start 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.516+0000 7f41a6dd4640 1 -- start start 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a00d30d0 0x7f41a014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 0x7f41a00ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41a015ed30 con 0x7f41a015e950 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41a00b7da0 con 0x7f41a00d30d0 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a6dd4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41a015eeb0 con 0x7f41a00d48c0 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a5371640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f419ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 0x7f41a00ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a5371640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33022/0 (socket says 172.21.15.181:33022) 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a4b70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a00d30d0 0x7f41a014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.517+0000 7f41a5371640 1 -- 172.21.15.181:0/1526288946 learned_addr learned my addr 172.21.15.181:0/1526288946 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 -- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a00d30d0 msgr2=0x7f41a014c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a00d30d0 0x7f41a014c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 -- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 msgr2=0x7f41a00ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 -- 172.21.15.181:0/1526288946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a014d050 con 0x7f41a00d48c0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f41a5371640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419ffff640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 0x7f41a00ca230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418c000f60 tx=0x7f418c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c017070 con 0x7f41a00d48c0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c005ce0 con 0x7f41a00d48c0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a00c9520 con 0x7f41a00d48c0 2024-03-29T04:09:42.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c007310 con 0x7f41a00d48c0 2024-03-29T04:09:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.518+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a014d2f0 con 0x7f41a00d48c0 2024-03-29T04:09:42.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.519+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a0075a10 con 0x7f41a00d48c0 2024-03-29T04:09:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.519+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f418c01a070 con 0x7f41a00d48c0 2024-03-29T04:09:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.520+0000 7f419d7fa640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 0x7f418406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.520+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f418c08acb0 con 0x7f41a00d48c0 2024-03-29T04:09:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.520+0000 7f41a4b70640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 0x7f418406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:42.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.520+0000 7f41a4b70640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 0x7f418406e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4194009560 tx=0x7f4194009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:42.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.524+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f418c058920 con 0x7f41a00d48c0 2024-03-29T04:09:42.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.735+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41a00b8720 con 0x7f41a00d48c0 2024-03-29T04:09:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.735+0000 7f419d7fa640 1 -- 172.21.15.181:0/1526288946 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f418c05cc20 con 0x7f41a00d48c0 2024-03-29T04:09:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:42.715 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:09:42.715 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.737+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 msgr2=0x7f418406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.737+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 0x7f418406e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4194009560 tx=0x7f4194009190 comp rx=0 tx=0).stop 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.737+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 msgr2=0x7f41a00ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.737+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 0x7f41a00ca230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418c000f60 tx=0x7f418c00c800 comp rx=0 tx=0).stop 2024-03-29T04:09:42.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 shutdown_connections 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41a015e950 0x7f41a00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f418406c530 0x7f418406e9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41a00d48c0 0x7f41a00ca230 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 --2- 172.21.15.181:0/1526288946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a00d30d0 0x7f41a014c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 >> 172.21.15.181:0/1526288946 conn(0x7f41a00676e0 msgr2=0x7f41a00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 shutdown_connections 2024-03-29T04:09:42.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:42.738+0000 7f41a6dd4640 1 -- 172.21.15.181:0/1526288946 wait complete. 2024-03-29T04:09:43.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:09:15.639711+0000', 'modified': '2024-03-29T04:09:31.705324+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6835/3068825067', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 3068825067}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3068825067}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-03-29T04:09:43.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:09:43.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T04:09:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.254+0000 7f06395e2640 1 Processor -- start 2024-03-29T04:09:43.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.254+0000 7f06395e2640 1 -- start start 2024-03-29T04:09:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f06395e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0634071440 0x7f06340cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f06395e2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0634071fe0 con 0x7f063415fba0 2024-03-29T04:09:43.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f06395e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0634071a20 con 0x7f06340cca60 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f06395e2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0634071d00 con 0x7f0634071440 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f0632ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f063415fba0 0x7f0634160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:34768/0 (socket says 172.21.15.181:34768) 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f0632ffd640 1 -- 172.21.15.181:0/767597801 learned_addr learned my addr 172.21.15.181:0/767597801 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f06327fc640 1 --2- 172.21.15.181:0/767597801 >> v2:172.21.15.187:3300/0 conn(0x7f0634071440 0x7f06340cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.255+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071590681 0 0) 0x7f0634071fe0 con 0x7f063415fba0 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f061c003520 con 0x7f063415fba0 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f06327fc640 1 -- 172.21.15.181:0/767597801 >> v1:172.21.15.187:6789/0 conn(0x7f06340cca60 legacy=0x7f06340b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f06327fc640 1 -- 172.21.15.181:0/767597801 >> v1:172.21.15.181:6789/0 conn(0x7f063415fba0 legacy=0x7f0634160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279938793 0 0) 0x7f0634071a20 con 0x7f06340cca60 2024-03-29T04:09:43.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f06327fc640 1 -- 172.21.15.181:0/767597801 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06340cabf0 con 0x7f0634071440 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.256+0000 7f06327fc640 1 --2- 172.21.15.181:0/767597801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0634071440 0x7f06340cb3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06280097a0 tx=0x7f062802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a526a75f84d4f6b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628034070 con 0x7f0634071440 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0628030610 con 0x7f0634071440 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f0631ffb640 1 -- 172.21.15.181:0/767597801 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280309c0 con 0x7f0634071440 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 -- 172.21.15.181:0/767597801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0634071440 msgr2=0x7f06340cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 --2- 172.21.15.181:0/767597801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0634071440 0x7f06340cb3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06280097a0 tx=0x7f062802b960 comp rx=0 tx=0).stop 2024-03-29T04:09:43.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 -- 172.21.15.181:0/767597801 shutdown_connections 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 --2- 172.21.15.181:0/767597801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0634071440 0x7f06340cb3f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 -- 172.21.15.181:0/767597801 >> 172.21.15.181:0/767597801 conn(0x7f063406d610 msgr2=0x7f063406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 -- 172.21.15.181:0/767597801 shutdown_connections 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.257+0000 7f06395e2640 1 -- 172.21.15.181:0/767597801 wait complete. 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.258+0000 7f06395e2640 1 Processor -- start 2024-03-29T04:09:43.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.258+0000 7f06395e2640 1 -- start start 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 0x7f063414e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 0x7f063414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06340b3630 con 0x7f0634071440 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06340cb930 con 0x7f06340cca60 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06395e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06340b37b0 con 0x7f063415fba0 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f0632ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f0632ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33034/0 (socket says 172.21.15.181:33034) 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f0632ffd640 1 -- 172.21.15.181:0/2067548710 learned_addr learned my addr 172.21.15.181:0/2067548710 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06327fc640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 0x7f063414e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.259+0000 7f06337fe640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 0x7f063414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 -- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 msgr2=0x7f063414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 0x7f063414d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 -- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 msgr2=0x7f063414e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 0x7f063414e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 -- 172.21.15.181:0/2067548710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063415b3e0 con 0x7f0634071440 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f06327fc640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 0x7f063414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f06337fe640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 0x7f063414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f0632ffd640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f062400caa0 tx=0x7f062400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f061b7fe640 1 -- 172.21.15.181:0/2067548710 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624017070 con 0x7f0634071440 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f061b7fe640 1 -- 172.21.15.181:0/2067548710 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062400ac00 con 0x7f0634071440 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06340c8110 con 0x7f0634071440 2024-03-29T04:09:43.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f061b7fe640 1 -- 172.21.15.181:0/2067548710 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624012400 con 0x7f0634071440 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.260+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06340c86d0 con 0x7f0634071440 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.261+0000 7f061b7fe640 1 -- 172.21.15.181:0/2067548710 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f062401a070 con 0x7f0634071440 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.262+0000 7f061b7fe640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 0x7f062006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.262+0000 7f061b7fe640 1 -- 172.21.15.181:0/2067548710 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f062408b690 con 0x7f0634071440 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.262+0000 7f06327fc640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 0x7f062006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.244 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.262+0000 7f06327fc640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 0x7f062006e9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0628036000 tx=0x7f062803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.263+0000 7f063c1ce740 1 Processor -- start 2024-03-29T04:09:43.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.263+0000 7f063c1ce740 1 -- start start 2024-03-29T04:09:43.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.250 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 0x5648a5897b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648a57d1020 con 0x5648a5894c40 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648a57795f0 con 0x5648a5894860 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5648a57ccfd0 con 0x5648a5867d40 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33040/0 (socket says 172.21.15.181:33040) 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06327fc640 1 -- 172.21.15.181:0/81426887 learned_addr learned my addr 172.21.15.181:0/81426887 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f0632ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 0x5648a5897b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 -- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 msgr2=0x5648a5867800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 -- 172.21.15.181:0/81426887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 msgr2=0x5648a58975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 -- 172.21.15.181:0/81426887 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a54e52d0 con 0x5648a5867d40 2024-03-29T04:09:43.251 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06327fc640 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f06337fe640 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 0x5648a5897b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f062c001b40 tx=0x7f062c00f5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d50d30055569f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.264+0000 7f0618ff9640 1 -- 172.21.15.181:0/81426887 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00fe60 con 0x5648a5867d40 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f0618ff9640 1 -- 172.21.15.181:0/81426887 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c018070 con 0x5648a5867d40 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f0618ff9640 1 -- 172.21.15.181:0/81426887 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c005310 con 0x5648a5867d40 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 -- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 msgr2=0x5648a5897b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 0x5648a5897b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f062c001b40 tx=0x7f062c00f5a0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 -- 172.21.15.181:0/81426887 shutdown_connections 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5867d40 0x5648a5897b30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 --2- 172.21.15.181:0/81426887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 -- 172.21.15.181:0/81426887 >> 172.21.15.181:0/81426887 conn(0x5648a5869a70 msgr2=0x5648a58b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 -- 172.21.15.181:0/81426887 shutdown_connections 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.265+0000 7f063c1ce740 1 -- 172.21.15.181:0/81426887 wait complete. 2024-03-29T04:09:43.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.266+0000 7f063c1ce740 1 -- start start 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.266+0000 7f063c1ce740 1 Processor -- start 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894c40 0x5648a58969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a589ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648a586c110 con 0x5648a5894860 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648a578c9c0 con 0x5648a5896f20 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5648a586e950 con 0x5648a5894c40 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 10 client.0 _ll_register_callbacks cb 0x5648a5770fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33052/0 (socket says 172.21.15.181:33052) 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 -- 172.21.15.181:0/3502374772 learned_addr learned my addr 172.21.15.181:0/3502374772 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f06337fe640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a589ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f06327fc640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894c40 0x5648a58969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 -- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894c40 msgr2=0x5648a58969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894c40 0x5648a58969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 -- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 msgr2=0x5648a589ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a589ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 -- 172.21.15.181:0/3502374772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a55af6b0 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0632ffd640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0624059340 tx=0x7f062405d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add5dc032f69e15e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062405d1b0 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f062405d350 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062c018070 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f0624058800 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 10 client.4863 fetch_fsmap learned FSMap version 9 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5648a58b2e10 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.267+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c018070 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f06240589a0 con 0x5648a5894860 2024-03-29T04:09:43.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 10 client.4863 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 20 client.4863 populate_metadata read hostname 'smithi181' 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a5895e70 0x5648a58b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 4 client.4863 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 --> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648a57a54e0 con 0x5648a5895e70 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f06327fc640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a5895e70 0x5648a58b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f0611ffb640 10 client.4863 ms_handle_connect on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.268+0000 7f06327fc640 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a5895e70 0x5648a58b2410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f018de0a20e3c08 server_cookie=d2a187770b0675f1 in_seq=0 out_seq=0 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f0611ffb640 1 -- 172.21.15.181:0/3502374772 <== mds.0 v2:172.21.15.187:6833/3068825067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5648a57a54e0 con 0x5648a5895e70 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f0611ffb640 10 client.4863 handle_command_reply: tid=0 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 client.4863 shutdown 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 msgr2=0x5648a58afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0624059340 tx=0x7f062405d7f0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 shutdown_connections 2024-03-29T04:09:43.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a589ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a5895e70 0x5648a58b2410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894c40 0x5648a58969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f0611ffb640 0 client.4863 ms_handle_reset on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3502374772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894860 0x5648a58afbd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 >> 172.21.15.181:0/3502374772 conn(0x5648a5867d40 msgr2=0x5648a586b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 shutdown_connections 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 1 -- 172.21.15.181:0/3502374772 wait complete. 2024-03-29T04:09:43.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.269+0000 7f063c1ce740 20 client.4863 trim_cache size 0 max 16384 2024-03-29T04:09:43.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.287+0000 7f063c1ce740 1 Processor -- start 2024-03-29T04:09:43.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.287+0000 7f063c1ce740 1 -- start start 2024-03-29T04:09:43.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5896f20 0x5648a586b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06341feb00 con 0x5648a5894c40 2024-03-29T04:09:43.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f063415a6e0 con 0x5648a5894860 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06341fe570 con 0x5648a5896f20 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33054/0 (socket says 172.21.15.181:33054) 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 -- 172.21.15.181:0/3721927381 learned_addr learned my addr 172.21.15.181:0/3721927381 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06337fe640 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5896f20 0x5648a586b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f0632ffd640 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 -- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5896f20 msgr2=0x5648a586b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5896f20 0x5648a586b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 -- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 msgr2=0x5648a5867280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 -- 172.21.15.181:0/3721927381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648a58b2390 con 0x5648a5894c40 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.288+0000 7f06327fc640 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06280098c0 tx=0x7f0628030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c6a77d3b241c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f0613fff640 1 -- 172.21.15.181:0/3721927381 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628034040 con 0x5648a5894c40 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f0613fff640 1 -- 172.21.15.181:0/3721927381 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062805c070 con 0x5648a5894c40 2024-03-29T04:09:43.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f0613fff640 1 -- 172.21.15.181:0/3721927381 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628057690 con 0x5648a5894c40 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 -- 172.21.15.181:0/3721927381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 msgr2=0x5648a586b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06280098c0 tx=0x7f0628030000 comp rx=0 tx=0).stop 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 -- 172.21.15.181:0/3721927381 shutdown_connections 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5896f20 0x5648a586b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a586b460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 --2- 172.21.15.181:0/3721927381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5894860 0x5648a5867280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 -- 172.21.15.181:0/3721927381 >> 172.21.15.181:0/3721927381 conn(0x5648a5867d40 msgr2=0x5648a58af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 -- 172.21.15.181:0/3721927381 shutdown_connections 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.289+0000 7f063c1ce740 1 -- 172.21.15.181:0/3721927381 wait complete. 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 -- start start 2024-03-29T04:09:43.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 Processor -- start 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894860 0x5648a58932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a58b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5648a5895cf0 con 0x5648a5894c40 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648a5895940 con 0x5648a5896f20 2024-03-29T04:09:43.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5648a5896310 con 0x5648a5894860 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.290+0000 7f063c1ce740 10 client.0 _ll_register_callbacks cb 0x7f0624058d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33060/0 (socket says 172.21.15.181:33060) 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 -- 172.21.15.181:0/1899663829 learned_addr learned my addr 172.21.15.181:0/1899663829 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06337fe640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a58b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0632ffd640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894860 0x5648a58932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 -- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894860 msgr2=0x5648a58932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894860 0x5648a58932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 -- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 msgr2=0x5648a58b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a58b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 -- 172.21.15.181:0/1899663829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648a592f650 con 0x5648a5894c40 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f06327fc640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0628059040 tx=0x7f0628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0942e417ec1837 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:09:43.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628034070 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0628004450 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062805c070 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f062806e050 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f063c1ce740 10 client.4869 fetch_fsmap learned FSMap version 9 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5648a592ee50 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.291+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062805c070 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f0628071020 con 0x5648a5894c40 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 10 client.4869 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 20 client.4869 populate_metadata read hostname 'smithi181' 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a58b5ba0 0x5648a586b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 4 client.4869 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 --> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] -- command(tid 0: {"prefix": "status"}) -- 0x5648a57de560 con 0x5648a58b5ba0 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0632ffd640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a58b5ba0 0x5648a586b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0611ffb640 10 client.4869 ms_handle_connect on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0632ffd640 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a58b5ba0 0x5648a586b360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6b5299404c37d1 server_cookie=abe8d179d481bf7 in_seq=0 out_seq=0 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0611ffb640 1 -- 172.21.15.181:0/1899663829 <== mds.0 v2:172.21.15.187:6833/3068825067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5648a57de560 con 0x5648a58b5ba0 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0611ffb640 10 client.4869 handle_command_reply: tid=0 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 client.4869 shutdown 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 msgr2=0x5648a58937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0628059040 tx=0x7f0628004730 comp rx=0 tx=0).stop 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 shutdown_connections 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648a5896f20 0x5648a58b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5648a5894c40 0x5648a58937f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:6833/3068825067,v1:172.21.15.187:6835/3068825067] conn(0x5648a58b5ba0 0x5648a586b360 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 --2- 172.21.15.181:0/1899663829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5648a5894860 0x5648a58932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f0611ffb640 0 client.4869 ms_handle_reset on v2:172.21.15.187:6833/3068825067 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 >> 172.21.15.181:0/1899663829 conn(0x5648a5867d40 msgr2=0x5648a5867280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 shutdown_connections 2024-03-29T04:09:43.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 1 -- 172.21.15.181:0/1899663829 wait complete. 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.292+0000 7f063c1ce740 20 client.4869 trim_cache size 0 max 16384 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "fab9a006-093b-45b9-834f-2720bd7aa224", 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4293, 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 26.992246508000001, 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 9, 2024-03-29T04:09:43.279 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 25, 2024-03-29T04:09:43.280 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 25, 2024-03-29T04:09:43.280 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 34.110730762000003 2024-03-29T04:09:43.280 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:09:43.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 msgr2=0x7f062006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 0x7f062006e9f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0628036000 tx=0x7f062803b040 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 msgr2=0x7f06340c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f062400caa0 tx=0x7f062400cf60 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 shutdown_connections 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f062006c530 0x7f062006e9f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f063415fba0 0x7f063414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06340cca60 0x7f063414e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 --2- 172.21.15.181:0/2067548710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0634071440 0x7f06340c7b30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 >> 172.21.15.181:0/2067548710 conn(0x7f063406d610 msgr2=0x7f063415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 shutdown_connections 2024-03-29T04:09:43.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.293+0000 7f06395e2640 1 -- 172.21.15.181:0/2067548710 wait complete. 2024-03-29T04:09:43.283 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T04:09:43.293 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T04:09:43.293 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T04:09:43.294 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T04:09:43.294 INFO:teuthology.run_tasks:Running task kclient... 2024-03-29T04:09:43.314 INFO:tasks.kclient:Mounting kernel clients... 2024-03-29T04:09:43.314 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T04:09:43.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.314 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:09:43.329 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-29T04:09:43.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.330 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T04:09:43.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-03-29T04:09:43.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.493 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:09:43.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-03-29T04:09:43.578 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2024-03-29T04:09:43.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:09:43.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.580 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:09:43.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.595 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-03-29T04:09:43.668 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-03-29T04:09:43.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:09:43.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.670 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:09:43.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:43.725 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-03-29T04:09:43.797 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-03-29T04:09:43.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:09:43.798 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-29T04:09:43.799 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:09:43.799 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:43.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:09:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.922+0000 7ff764414640 1 Processor -- start 2024-03-29T04:09:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.922+0000 7ff764414640 1 -- start start 2024-03-29T04:09:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff764414640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff764414640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff75c06b580 0x7ff75c06ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff764414640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75c0ce810 con 0x7ff75c1451b0 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff764414640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff75c145590 con 0x7ff75c06b580 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff764414640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c160440 con 0x7ff75c145d10 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:45480/0 (socket says 172.21.15.181:45480) 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7629b1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff75c06b580 0x7ff75c06ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 -- 172.21.15.181:0/2774296020 learned_addr learned my addr 172.21.15.181:0/2774296020 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 -- 172.21.15.181:0/2774296020 >> v2:172.21.15.187:3300/0 conn(0x7ff75c06b580 msgr2=0x7ff75c06ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 --2- 172.21.15.181:0/2774296020 >> v2:172.21.15.187:3300/0 conn(0x7ff75c06b580 0x7ff75c06ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 -- 172.21.15.181:0/2774296020 >> v1:172.21.15.187:6789/0 conn(0x7ff75c1451b0 legacy=0x7ff75c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.923+0000 7ff7619af640 1 -- 172.21.15.181:0/2774296020 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c0ce250 con 0x7ff75c145d10 2024-03-29T04:09:43.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff7619af640 1 --2- 172.21.15.181:0/2774296020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff750009870 tx=0x7ff75002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e07c82529ac7264 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff7611ae640 1 -- 172.21.15.181:0/2774296020 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035020 con 0x7ff75c145d10 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff7611ae640 1 -- 172.21.15.181:0/2774296020 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff750031640 con 0x7ff75c145d10 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff7611ae640 1 -- 172.21.15.181:0/2774296020 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75002d040 con 0x7ff75c145d10 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff764414640 1 -- 172.21.15.181:0/2774296020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 msgr2=0x7ff75c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.924+0000 7ff764414640 1 --2- 172.21.15.181:0/2774296020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff750009870 tx=0x7ff75002b8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 -- 172.21.15.181:0/2774296020 shutdown_connections 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 --2- 172.21.15.181:0/2774296020 >> v2:172.21.15.187:3300/0 conn(0x7ff75c06b580 0x7ff75c06ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 --2- 172.21.15.181:0/2774296020 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c0cd8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 -- 172.21.15.181:0/2774296020 >> 172.21.15.181:0/2774296020 conn(0x7ff75c0b2e10 msgr2=0x7ff75c0b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 -- 172.21.15.181:0/2774296020 shutdown_connections 2024-03-29T04:09:43.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.925+0000 7ff764414640 1 -- 172.21.15.181:0/2774296020 wait complete. 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 Processor -- start 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 -- start start 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c0cde20 con 0x7ff75c1451b0 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1600d0 con 0x7ff75c06b580 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.926+0000 7ff764414640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c0cdfa0 con 0x7ff75c145d10 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7629b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7621b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33074/0 (socket says 172.21.15.181:33074) 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 -- 172.21.15.181:0/2734695223 learned_addr learned my addr 172.21.15.181:0/2734695223 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:43.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7629b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c15ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:45492/0 (socket says 172.21.15.181:45492) 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7621b0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:58198/0 (socket says 172.21.15.181:58198) 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 -- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 msgr2=0x7ff75c15ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c15ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 -- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 msgr2=0x7ff75c0b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 -- 172.21.15.181:0/2734695223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c0cac50 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7621b0640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff7619af640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff750000c00 tx=0x7ff7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750040070 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7500399f0 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c0ca970 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035040 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.927+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c0b4fd0 con 0x7ff75c1451b0 2024-03-29T04:09:43.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.928+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c075a10 con 0x7ff75c1451b0 2024-03-29T04:09:43.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.929+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff75004b060 con 0x7ff75c1451b0 2024-03-29T04:09:43.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.929+0000 7ff74effd640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 0x7ff74006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:43.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.929+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff7500ad080 con 0x7ff75c1451b0 2024-03-29T04:09:43.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.929+0000 7ff7621b0640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 0x7ff74006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:43.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.929+0000 7ff7621b0640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 0x7ff74006e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff758001b70 tx=0x7ff758009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:43.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:43.933+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff75007ada0 con 0x7ff75c1451b0 2024-03-29T04:09:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.144+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff75c144960 con 0x7ff75c1451b0 2024-03-29T04:09:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.144+0000 7ff74effd640 1 -- 172.21.15.181:0/2734695223 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff75007f0a0 con 0x7ff75c1451b0 2024-03-29T04:09:44.124 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 msgr2=0x7ff74006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 0x7ff74006e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff758001b70 tx=0x7ff758009040 comp rx=0 tx=0).stop 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 msgr2=0x7ff75c0cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff750000c00 tx=0x7ff7500046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 shutdown_connections 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff74006c530 0x7ff74006e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff75c145d10 0x7ff75c15ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff75c1451b0 0x7ff75c0cb110 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 --2- 172.21.15.181:0/2734695223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff75c06b580 0x7ff75c0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 >> 172.21.15.181:0/2734695223 conn(0x7ff75c0b2e10 msgr2=0x7ff75c13fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 shutdown_connections 2024-03-29T04:09:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44.146+0000 7ff764414640 1 -- 172.21.15.181:0/2734695223 wait complete. 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:09:44.137 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:09:44.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.137 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-03-29T04:09:44.152 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:44.153 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-03-29T04:09:44.154 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:44.154 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:09:44.154 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:44.154 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:44.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:09:44.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2024-03-29T04:09:44.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2024-03-29T04:09:44.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:09:44.155 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:44.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:44.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:44.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.300 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:09:44.326 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-03-29T04:09:44.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.328 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-03-29T04:09:44.386 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-03-29T04:09:44.386 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:09:44.386 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:09:44.386 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:09:44.387 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:09:44.387 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:44.387 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:44.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:09:44.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:09:44.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:09:44.388 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:44.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:44.516 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:44.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.522 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:09:44.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.538 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-03-29T04:09:44.594 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:44.594 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:44.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:09:44.594 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T04:09:44.595 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:44.681 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:44.709 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:44.715 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T04:09:44.716 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:44.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:44.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:44.840 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:44.840 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:44.840 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-03-29T04:09:44.840 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:09:44.840 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:44.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:44.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:44.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:44.922 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:44.937 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:09:44.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:44.937 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:44.992 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-29T04:09:44.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:44.992 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:45.077 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:45.077 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:45.185 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:45.185 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:45.185 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:09:45.185 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 0@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:09:45.185 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:09:45.186 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-29T04:09:45.186 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:09:45.186 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:09:45.186 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:09:45.186 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:09:45.187 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:09:45.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:45.188 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:09:45.214 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:45.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:45.309 INFO:teuthology.orchestra.run.smithi181.stdout:client4637 2024-03-29T04:09:45.310 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:45.310 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:45.325 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:45.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:45.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:45.422 INFO:teuthology.orchestra.run.smithi181.stdout:client4637 2024-03-29T04:09:45.422 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:45.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:45.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:45.478 INFO:teuthology.orchestra.run.smithi181.stdout:client4637 2024-03-29T04:09:45.478 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fab9a006-093b-45b9-834f-2720bd7aa224.client4637/status') 2024-03-29T04:09:45.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:45.540 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4637 (3)172.21.15.181:0/1063501831 2024-03-29T04:09:45.540 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2024-03-29T04:09:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2024-03-29T04:09:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2024-03-29T04:09:45.540 INFO:teuthology.orchestra.run.smithi181.stderr:71 bytes copied, 2.1685e-05 s, 3.3 MB/s 2024-03-29T04:09:45.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:45.547 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-29T04:09:45.547 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:09:45.547 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:09:45.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:09:45.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.668+0000 7f5e3f1af640 1 Processor -- start 2024-03-29T04:09:45.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.668+0000 7f5e3f1af640 1 -- start start 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5e381445e0 0x7f5e380cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e380cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e380727a0 0x7f5e380cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5e380cd610 con 0x7f5e380727a0 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5e381449c0 con 0x7f5e381445e0 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3f1af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e380cf5b0 con 0x7f5e38072340 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e380727a0 0x7f5e380cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.649 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e380727a0 0x7f5e380cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33088/0 (socket says 172.21.15.181:33088) 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 -- 172.21.15.181:0/899665920 learned_addr learned my addr 172.21.15.181:0/899665920 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e37fff640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e380cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3cf4b640 1 --2- 172.21.15.181:0/899665920 >> v2:172.21.15.187:3300/0 conn(0x7f5e381445e0 0x7f5e380cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 -- 172.21.15.181:0/899665920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 msgr2=0x7f5e380cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e380cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 -- 172.21.15.181:0/899665920 >> v2:172.21.15.187:3300/0 conn(0x7f5e381445e0 msgr2=0x7f5e380cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/899665920 >> v2:172.21.15.187:3300/0 conn(0x7f5e381445e0 0x7f5e380cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.669+0000 7f5e3d74c640 1 -- 172.21.15.181:0/899665920 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e38144e10 con 0x7f5e380727a0 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.670+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e380727a0 0x7f5e380cbbd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e380d10c0 tx=0x7f5e3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5404da3ec51a472 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.670+0000 7f5e377fe640 1 -- 172.21.15.181:0/899665920 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3002ca70 con 0x7f5e380727a0 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.670+0000 7f5e377fe640 1 -- 172.21.15.181:0/899665920 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3002cc10 con 0x7f5e380727a0 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.670+0000 7f5e377fe640 1 -- 172.21.15.181:0/899665920 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300343f0 con 0x7f5e380727a0 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 -- 172.21.15.181:0/899665920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e380727a0 msgr2=0x7f5e380cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e380727a0 0x7f5e380cbbd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e380d10c0 tx=0x7f5e3002bed0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 -- 172.21.15.181:0/899665920 shutdown_connections 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e380727a0 0x7f5e380cbbd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/899665920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e380cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/899665920 >> v2:172.21.15.187:3300/0 conn(0x7f5e381445e0 0x7f5e380cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 -- 172.21.15.181:0/899665920 >> 172.21.15.181:0/899665920 conn(0x7f5e380b1c90 msgr2=0x7f5e380b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:45.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 -- 172.21.15.181:0/899665920 shutdown_connections 2024-03-29T04:09:45.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.671+0000 7f5e3f1af640 1 -- 172.21.15.181:0/899665920 wait complete. 2024-03-29T04:09:45.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.672+0000 7f5e3f1af640 1 Processor -- start 2024-03-29T04:09:45.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.672+0000 7f5e3f1af640 1 -- start start 2024-03-29T04:09:45.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e381f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e380727a0 0x7f5e381f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e380cad60 con 0x7f5e381445e0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3815edb0 con 0x7f5e380727a0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3f1af640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e380cb040 con 0x7f5e38072340 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33096/0 (socket says 172.21.15.181:33096) 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 -- 172.21.15.181:0/3374694054 learned_addr learned my addr 172.21.15.181:0/3374694054 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e37fff640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e380727a0 0x7f5e381f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3cf4b640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e381f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 -- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 msgr2=0x7f5e381f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e381f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 -- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e380727a0 msgr2=0x7f5e381f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e380727a0 0x7f5e381f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.673+0000 7f5e3d74c640 1 -- 172.21.15.181:0/3374694054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e381f3430 con 0x7f5e381445e0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e3d74c640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5e30036040 tx=0x7f5e30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:45.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30043020 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3003e450 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e381f3710 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30009510 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.674+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381f3cd0 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.675+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e38075a10 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.675+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5e30046020 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.676+0000 7f5e357fa640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 0x7f5e2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.676+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e300adf60 con 0x7f5e381445e0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.676+0000 7f5e3cf4b640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 0x7f5e2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:45.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.676+0000 7f5e3cf4b640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 0x7f5e2406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e38143170 tx=0x7f5e28009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:45.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.680+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5e3004d030 con 0x7f5e381445e0 2024-03-29T04:09:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.892+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e380cd790 con 0x7f5e381445e0 2024-03-29T04:09:45.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.892+0000 7f5e357fa640 1 -- 172.21.15.181:0/3374694054 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f5e3007bbf0 con 0x7f5e381445e0 2024-03-29T04:09:45.872 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 msgr2=0x7f5e2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 0x7f5e2406e9f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e38143170 tx=0x7f5e28009190 comp rx=0 tx=0).stop 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 msgr2=0x7f5e381f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5e30036040 tx=0x7f5e30004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 shutdown_connections 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e381445e0 0x7f5e381f2d20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e380727a0 0x7f5e381f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5e2406c530 0x7f5e2406e9f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 --2- 172.21.15.181:0/3374694054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e38072340 0x7f5e381f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 >> 172.21.15.181:0/3374694054 conn(0x7f5e380b1c90 msgr2=0x7f5e380b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 shutdown_connections 2024-03-29T04:09:45.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:45.894+0000 7f5e3f1af640 1 -- 172.21.15.181:0/3374694054 wait complete. 2024-03-29T04:09:45.884 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:09:45.885 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:09:45.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:45.885 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-03-29T04:09:45.901 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 6e:d7:dc:da:09:7c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::9851:47ff:fed2:ccc4/64 scope link tentative 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.902 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:09:45.903 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 6e:d7:dc:da:09:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:09:45.903 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::6cd7:dcff:feda:97c/64 scope link tentative 2024-03-29T04:09:45.903 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:45.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:45.903 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:09:45.967 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-03-29T04:09:45.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:45.968 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-03-29T04:09:46.024 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-03-29T04:09:46.024 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:09:46.024 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:09:46.024 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:09:46.024 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:09:46.025 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:46.112 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:46.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:46.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.159 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:09:46.174 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:09:46.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.174 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-03-29T04:09:46.229 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T04:09:46.229 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:46.229 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:46.229 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:09:46.229 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T04:09:46.230 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:46.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:46.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:46.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:09:46.379 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c2:e1:e6:70:d3:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c0e1:e6ff:fe70:d312/64 scope link tentative 2024-03-29T04:09:46.380 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c2:e1:e6:70:d3:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:09:46.448 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:09:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c0e1:e6ff:fe70:d312/64 scope link tentative 2024-03-29T04:09:46.449 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:46.450 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T04:09:46.450 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:46.450 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:46.450 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T04:09:46.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:09:46.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T04:09:46.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T04:09:46.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T04:09:46.451 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:46.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:46.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:46.615 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:46.615 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:09:46.615 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2024-03-29T04:09:46.615 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:09:46.615 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:09:46.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:46.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:46.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:46.698 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T04:09:46.713 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T04:09:46.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:46.713 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:09:46.767 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-29T04:09:46.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.768 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:46.853 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:46.853 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 2@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:09:46.860 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:09:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:09:46.861 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:09:46.862 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:09:46.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:46.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T04:09:46.889 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:46.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:46.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:46.985 INFO:teuthology.orchestra.run.smithi181.stdout:client4846 2024-03-29T04:09:46.985 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:46.985 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:47.000 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:47.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:47.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:47.096 INFO:teuthology.orchestra.run.smithi181.stdout:client4846 2024-03-29T04:09:47.096 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:47.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:47.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:47.152 INFO:teuthology.orchestra.run.smithi181.stdout:client4846 2024-03-29T04:09:47.152 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fab9a006-093b-45b9-834f-2720bd7aa224.client4846/status') 2024-03-29T04:09:47.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:47.213 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4846 (3)172.21.15.181:0/2303348541 2024-03-29T04:09:47.213 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2024-03-29T04:09:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2024-03-29T04:09:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2024-03-29T04:09:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:71 bytes copied, 1.9901e-05 s, 3.6 MB/s 2024-03-29T04:09:47.214 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:47.220 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-29T04:09:47.221 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:09:47.221 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:47.221 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:09:47.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.343+0000 7f8e5436d640 1 Processor -- start 2024-03-29T04:09:47.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.343+0000 7f8e5436d640 1 -- start start 2024-03-29T04:09:47.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8e4c145090 0x7f8e4c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c1607e0 con 0x7f8e4c160370 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0b5f80 con 0x7f8e4c145090 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5436d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0b6260 con 0x7f8e4c0b5aa0 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e5290a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.344+0000 7f8e52109640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8e4c145090 0x7f8e4c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8e4c145090 0x7f8e4c0b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:58224/0 (socket says 172.21.15.181:58224) 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 -- 172.21.15.181:0/166913825 learned_addr learned my addr 172.21.15.181:0/166913825 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e5290a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33110/0 (socket says 172.21.15.181:33110) 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e51908640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 -- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 msgr2=0x7f8e4c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c15fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 -- 172.21.15.181:0/166913825 >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 msgr2=0x7f8e4c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 --2- 172.21.15.181:0/166913825 >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e52109640 1 -- 172.21.15.181:0/166913825 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c0b6540 con 0x7f8e4c145090 2024-03-29T04:09:47.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.345+0000 7f8e5290a640 1 --2- 172.21.15.181:0/166913825 >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e52109640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c145090 0x7f8e4c0b1720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0681c0 tx=0x7f8e4802b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43031387f1e34700 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e51107640 1 -- 172.21.15.181:0/166913825 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4802e070 con 0x7f8e4c145090 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e51107640 1 -- 172.21.15.181:0/166913825 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48037070 con 0x7f8e4c145090 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e51107640 1 -- 172.21.15.181:0/166913825 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48031a00 con 0x7f8e4c145090 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 -- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c145090 msgr2=0x7f8e4c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c145090 0x7f8e4c0b1720 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0681c0 tx=0x7f8e4802b880 comp rx=0 tx=0).stop 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 -- 172.21.15.181:0/166913825 shutdown_connections 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 --2- 172.21.15.181:0/166913825 >> v2:172.21.15.181:3300/0 conn(0x7f8e4c160370 0x7f8e4c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 --2- 172.21.15.181:0/166913825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c145090 0x7f8e4c0b1720 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 -- 172.21.15.181:0/166913825 >> 172.21.15.181:0/166913825 conn(0x7f8e4c06f8b0 msgr2=0x7f8e4c06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:47.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 -- 172.21.15.181:0/166913825 shutdown_connections 2024-03-29T04:09:47.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.346+0000 7f8e5436d640 1 -- 172.21.15.181:0/166913825 wait complete. 2024-03-29T04:09:47.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.347+0000 7f8e5436d640 1 Processor -- start 2024-03-29T04:09:47.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.347+0000 7f8e5436d640 1 -- start start 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 0x7f8e4c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c146000 con 0x7f8e4c145090 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d0c30 con 0x7f8e4c160370 2024-03-29T04:09:47.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5436d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c145470 con 0x7f8e4c0b5aa0 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e51908640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e51908640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33114/0 (socket says 172.21.15.181:33114) 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e51908640 1 -- 172.21.15.181:0/3640116221 learned_addr learned my addr 172.21.15.181:0/3640116221 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e5290a640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 0x7f8e4c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.348+0000 7f8e52109640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 -- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 msgr2=0x7f8e4c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 -- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 msgr2=0x7f8e4c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 0x7f8e4c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 -- 172.21.15.181:0/3640116221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c13fc10 con 0x7f8e4c145090 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e5290a640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 0x7f8e4c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e51908640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00e8a0 tx=0x7f8e3c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c017070 con 0x7f8e4c145090 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00ee90 con 0x7f8e4c145090 2024-03-29T04:09:47.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c13fef0 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00cb80 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.349+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1404b0 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.350+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c075a10 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.350+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8e3c01a070 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.351+0000 7f8e42ffd640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 0x7f8e3006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.351+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8e3c08ba80 con 0x7f8e4c145090 2024-03-29T04:09:47.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.351+0000 7f8e52109640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 0x7f8e3006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:47.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.351+0000 7f8e52109640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 0x7f8e3006ea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e48036000 tx=0x7f8e4803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:47.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.355+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8e3c059720 con 0x7f8e4c145090 2024-03-29T04:09:47.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.570+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e4c0b48e0 con 0x7f8e4c145090 2024-03-29T04:09:47.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.571+0000 7f8e42ffd640 1 -- 172.21.15.181:0/3640116221 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f8e3c05da20 con 0x7f8e4c145090 2024-03-29T04:09:47.550 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:09:47.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 msgr2=0x7f8e3006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:47.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 0x7f8e3006ea20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e48036000 tx=0x7f8e4803d040 comp rx=0 tx=0).stop 2024-03-29T04:09:47.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 msgr2=0x7f8e4c13f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:47.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00e8a0 tx=0x7f8e3c00eca0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 shutdown_connections 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e4c160370 0x7f8e4c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e4c145090 0x7f8e4c13f3c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e3006c560 0x7f8e3006ea20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 --2- 172.21.15.181:0/3640116221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e4c0b5aa0 0x7f8e4c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.572+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 >> 172.21.15.181:0/3640116221 conn(0x7f8e4c06f8b0 msgr2=0x7f8e4c06ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.573+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 shutdown_connections 2024-03-29T04:09:47.553 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:47.573+0000 7f8e5436d640 1 -- 172.21.15.181:0/3640116221 wait complete. 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:09:47.563 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:09:47.564 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:09:47.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:47.564 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-03-29T04:09:47.578 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42707sec preferred_lft 42707sec 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-03-29T04:09:47.580 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:47.580 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T04:09:47.580 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:47.580 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:47.580 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-03-29T04:09:47.580 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-03-29T04:09:47.580 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-03-29T04:09:47.581 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T04:09:47.581 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:47.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:47.844 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:47.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:47.851 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:09:47.878 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-03-29T04:09:47.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:47.881 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-03-29T04:09:47.948 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-03-29T04:09:47.948 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:09:47.948 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:09:47.948 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:09:47.948 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:09:47.949 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:48.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:48.231 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:48.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:48.239 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:09:48.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:48.254 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-03-29T04:09:48.309 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:48.309 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:48.310 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:09:48.310 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T04:09:48.310 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:48.399 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:48.428 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:48.434 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T04:09:48.434 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T04:09:48.435 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T04:09:48.435 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:48.483 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:48.568 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:48.576 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:48.576 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:48.576 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-03-29T04:09:48.577 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T04:09:48.577 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:48.624 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:48.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:48.660 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:48.660 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:48.674 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:09:48.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:48.675 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:48.729 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-29T04:09:48.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:48.729 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:48.815 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:48.816 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:48.925 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with new device syntax: 1@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-29T04:09:48.926 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:09:48.927 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:09:48.927 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:09:48.927 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:09:48.928 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:09:48.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:48.929 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:09:48.955 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:49.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:49.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:49.058 INFO:teuthology.orchestra.run.smithi187.stdout:client4887 2024-03-29T04:09:49.058 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:09:49.059 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:49.075 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:49.164 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:49.165 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:49.173 INFO:teuthology.orchestra.run.smithi187.stdout:client4887 2024-03-29T04:09:49.173 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:49.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:49.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:49.228 INFO:teuthology.orchestra.run.smithi187.stdout:client4887 2024-03-29T04:09:49.229 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fab9a006-093b-45b9-834f-2720bd7aa224.client4887/status') 2024-03-29T04:09:49.275 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:49.288 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4887 (3)172.21.15.187:0/1221938792 2024-03-29T04:09:49.288 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-03-29T04:09:49.288 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-03-29T04:09:49.288 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-03-29T04:09:49.288 INFO:teuthology.orchestra.run.smithi187.stderr:71 bytes copied, 2.0968e-05 s, 3.4 MB/s 2024-03-29T04:09:49.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:49.295 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-29T04:09:49.295 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T04:09:49.295 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:09:49.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:09:49.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.428+0000 7fbea189a640 1 Processor -- start 2024-03-29T04:09:49.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.428+0000 7fbea189a640 1 -- start start 2024-03-29T04:09:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbea189a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe9c141530 0x7fbe9c071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbea189a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbe9c162030 0x7fbe9c1624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbea189a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9c141910 con 0x7fbe9c0b5240 2024-03-29T04:09:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbea189a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c071e60 con 0x7fbe9c141530 2024-03-29T04:09:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbea189a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c162af0 con 0x7fbe9c162030 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbe9affd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe9c141530 0x7fbe9c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbe9b7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe9c0b5240 0x7fbe9c141c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36970/0 (socket says 172.21.15.181:36970) 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbe9affd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe9c141530 0x7fbe9c071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50886/0 (socket says 172.21.15.181:50886) 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbe9b7fe640 1 -- 172.21.15.181:0/3990754787 learned_addr learned my addr 172.21.15.181:0/3990754787 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.429+0000 7fbe9a7fc640 1 --2- 172.21.15.181:0/3990754787 >> v2:172.21.15.187:3300/0 conn(0x7fbe9c162030 0x7fbe9c1624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe9affd640 1 -- 172.21.15.181:0/3990754787 >> v2:172.21.15.187:3300/0 conn(0x7fbe9c162030 msgr2=0x7fbe9c1624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe9affd640 1 --2- 172.21.15.181:0/3990754787 >> v2:172.21.15.187:3300/0 conn(0x7fbe9c162030 0x7fbe9c1624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe9affd640 1 -- 172.21.15.181:0/3990754787 >> v1:172.21.15.181:6789/0 conn(0x7fbe9c0b5240 legacy=0x7fbe9c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe9affd640 1 -- 172.21.15.181:0/3990754787 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c162dd0 con 0x7fbe9c141530 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe99ffb640 1 -- 172.21.15.181:0/3990754787 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763065986 0 0) 0x7fbe9c141910 con 0x7fbe9c0b5240 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.430+0000 7fbe9affd640 1 --2- 172.21.15.181:0/3990754787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c141530 0x7fbe9c071920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c161f70 tx=0x7fbe8c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5e2750cd583a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbe99ffb640 1 -- 172.21.15.181:0/3990754787 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c039070 con 0x7fbe9c141530 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbe99ffb640 1 -- 172.21.15.181:0/3990754787 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c02cbf0 con 0x7fbe9c141530 2024-03-29T04:09:49.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbe99ffb640 1 -- 172.21.15.181:0/3990754787 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c034440 con 0x7fbe9c141530 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 -- 172.21.15.181:0/3990754787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c141530 msgr2=0x7fbe9c071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 --2- 172.21.15.181:0/3990754787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c141530 0x7fbe9c071920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c161f70 tx=0x7fbe8c02be40 comp rx=0 tx=0).stop 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 -- 172.21.15.181:0/3990754787 shutdown_connections 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 --2- 172.21.15.181:0/3990754787 >> v2:172.21.15.187:3300/0 conn(0x7fbe9c162030 0x7fbe9c1624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 --2- 172.21.15.181:0/3990754787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c141530 0x7fbe9c071920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 -- 172.21.15.181:0/3990754787 >> 172.21.15.181:0/3990754787 conn(0x7fbe9c06f8b0 msgr2=0x7fbe9c06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 -- 172.21.15.181:0/3990754787 shutdown_connections 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.431+0000 7fbea189a640 1 -- 172.21.15.181:0/3990754787 wait complete. 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.432+0000 7fbea189a640 1 Processor -- start 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 -- start start 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9c0b5240 0x7fbe9c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe9c141530 0x7fbe9c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c0b1950 con 0x7fbe9c162030 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c071400 con 0x7fbe9c0b5240 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbea189a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c142320 con 0x7fbe9c141530 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.433+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9c0b5240 0x7fbe9c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50898/0 (socket says 172.21.15.181:50898) 2024-03-29T04:09:49.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9a7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe9c141530 0x7fbe9c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 -- 172.21.15.181:0/640865751 learned_addr learned my addr 172.21.15.181:0/640865751 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 -- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe9c141530 msgr2=0x7fbe9c158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe9c141530 0x7fbe9c158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 -- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9c0b5240 msgr2=0x7fbe9c1f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9c0b5240 0x7fbe9c1f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 -- 172.21.15.181:0/640865751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c159640 con 0x7fbe9c162030 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe9b7fe640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c450 tx=0x7fbe9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940074e0 con 0x7fbe9c162030 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940047e0 con 0x7fbe9c162030 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94004bd0 con 0x7fbe9c162030 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c159920 con 0x7fbe9c162030 2024-03-29T04:09:49.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.434+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c15a1c0 con 0x7fbe9c162030 2024-03-29T04:09:49.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.435+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c159af0 con 0x7fbe9c162030 2024-03-29T04:09:49.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.436+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe94011070 con 0x7fbe9c162030 2024-03-29T04:09:49.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.436+0000 7fbe83fff640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 0x7fbe8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:49.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.436+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbe9408b4e0 con 0x7fbe9c162030 2024-03-29T04:09:49.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.436+0000 7fbe9affd640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 0x7fbe8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:49.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.437+0000 7fbe9affd640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 0x7fbe8806e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c036000 tx=0x7fbe8c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:49.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.441+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe94059170 con 0x7fbe9c162030 2024-03-29T04:09:49.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.652+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbe9c075a10 con 0x7fbe9c162030 2024-03-29T04:09:49.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.653+0000 7fbe83fff640 1 -- 172.21.15.181:0/640865751 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fbe9405d470 con 0x7fbe9c162030 2024-03-29T04:09:49.632 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 msgr2=0x7fbe8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 0x7fbe8806e9f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c036000 tx=0x7fbe8c030000 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 msgr2=0x7fbe9c0cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c450 tx=0x7fbe9400c910 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 shutdown_connections 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe9c162030 0x7fbe9c0cb6c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbe8806c530 0x7fbe8806e9f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe9c141530 0x7fbe9c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 --2- 172.21.15.181:0/640865751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe9c0b5240 0x7fbe9c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 >> 172.21.15.181:0/640865751 conn(0x7fbe9c06f8b0 msgr2=0x7fbe9c1426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 shutdown_connections 2024-03-29T04:09:49.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:49.655+0000 7fbea189a640 1 -- 172.21.15.181:0/640865751 wait complete. 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T04:09:49.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T04:09:49.646 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T04:09:49.646 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:09:49.646 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:09:49.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:49.646 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:49.662 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether e2:ad:e4:80:6e:c6 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::78a9:f5ff:fe74:e7b6/64 scope link 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.663 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:09:49.664 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether e2:ad:e4:80:6e:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:09:49.664 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::e0ad:e4ff:fe80:6ec6/64 scope link tentative 2024-03-29T04:09:49.664 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:49.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:49.664 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:09:49.728 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-03-29T04:09:49.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:49.729 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-03-29T04:09:49.784 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-03-29T04:09:49.785 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:09:49.785 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:09:49.785 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:09:49.785 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:09:49.785 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:49.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:49.914 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:49.921 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:09:49.935 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:09:49.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:49.936 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-03-29T04:09:49.991 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T04:09:49.991 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:49.991 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:49.991 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:09:49.991 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T04:09:49.991 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:50.079 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:50.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:50.113 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:09:50.139 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:50.139 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:50.139 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:50.139 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.139 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 82:76:64:5e:d4:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::8076:64ff:fe5e:d4c6/64 scope link 2024-03-29T04:09:50.140 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:50.141 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 82:76:64:5e:d4:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.208 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::8076:64ff:fe5e:d4c6/64 scope link 2024-03-29T04:09:50.209 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:09:50.209 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T04:09:50.210 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:50.297 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:50.369 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.375 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:09:50.375 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:09:50.375 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-03-29T04:09:50.375 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T04:09:50.375 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:09:50.423 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:50.452 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:50.459 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T04:09:50.474 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T04:09:50.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:09:50.475 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:09:50.530 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-29T04:09:50.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:50.530 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:50.616 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:50.616 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:50.623 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with new device syntax: 3@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:09:50.624 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:09:50.625 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:09:50.625 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:09:50.626 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:09:50.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:09:50.626 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T04:09:50.652 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:50.740 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:50.742 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.747 INFO:teuthology.orchestra.run.smithi187.stdout:client4655 2024-03-29T04:09:50.748 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:09:50.748 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:50.763 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:50.856 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:50.857 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.863 INFO:teuthology.orchestra.run.smithi187.stdout:client4655 2024-03-29T04:09:50.863 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:09:50.924 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:09:50.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:50.932 INFO:teuthology.orchestra.run.smithi187.stdout:client4655 2024-03-29T04:09:50.933 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fab9a006-093b-45b9-834f-2720bd7aa224.client4655/status') 2024-03-29T04:09:50.981 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:09:50.993 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4655 (3)192.168.0.2:0/2925219342 2024-03-29T04:09:50.993 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-03-29T04:09:50.993 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-03-29T04:09:50.994 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-03-29T04:09:50.994 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 2.0588e-05 s, 3.4 MB/s 2024-03-29T04:09:50.994 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:09:51.001 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T04:09:51.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:51.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.150+0000 7fc687814640 1 Processor -- start 2024-03-29T04:09:51.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.150+0000 7fc687814640 1 -- start start 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc687814640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc680145c60 0x7fc680071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc687814640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc680160830 0x7fc6800ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc687814640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc680146040 con 0x7fc680071a60 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc687814640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc680071ed0 con 0x7fc680145c60 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc687814640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc680160e50 con 0x7fc680160830 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc680145c60 0x7fc680071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc680145c60 0x7fc680071520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50908/0 (socket says 172.21.15.181:50908) 2024-03-29T04:09:51.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 -- 172.21.15.181:0/1635735198 learned_addr learned my addr 172.21.15.181:0/1635735198 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc685db1640 1 --2- 172.21.15.181:0/1635735198 >> v2:172.21.15.187:3300/0 conn(0x7fc680160830 0x7fc6800ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 -- 172.21.15.181:0/1635735198 >> v2:172.21.15.187:3300/0 conn(0x7fc680160830 msgr2=0x7fc6800ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:51.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 --2- 172.21.15.181:0/1635735198 >> v2:172.21.15.187:3300/0 conn(0x7fc680160830 0x7fc6800ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 -- 172.21.15.181:0/1635735198 >> v1:172.21.15.187:6789/0 conn(0x7fc680071a60 legacy=0x7fc680160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.151+0000 7fc6855b0640 1 -- 172.21.15.181:0/1635735198 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68015d8b0 con 0x7fc680145c60 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.152+0000 7fc6855b0640 1 --2- 172.21.15.181:0/1635735198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680145c60 0x7fc680071520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6800b2960 tx=0x7fc67c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826a01cf7193c426 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.152+0000 7fc677fff640 1 -- 172.21.15.181:0/1635735198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c035020 con 0x7fc680145c60 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.152+0000 7fc677fff640 1 -- 172.21.15.181:0/1635735198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc67c031870 con 0x7fc680145c60 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.152+0000 7fc677fff640 1 -- 172.21.15.181:0/1635735198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c031c40 con 0x7fc680145c60 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 -- 172.21.15.181:0/1635735198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680145c60 msgr2=0x7fc680071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 --2- 172.21.15.181:0/1635735198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680145c60 0x7fc680071520 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6800b2960 tx=0x7fc67c02b910 comp rx=0 tx=0).stop 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 -- 172.21.15.181:0/1635735198 shutdown_connections 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 --2- 172.21.15.181:0/1635735198 >> v2:172.21.15.187:3300/0 conn(0x7fc680160830 0x7fc6800ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 --2- 172.21.15.181:0/1635735198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680145c60 0x7fc680071520 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 -- 172.21.15.181:0/1635735198 >> 172.21.15.181:0/1635735198 conn(0x7fc6800b1c90 msgr2=0x7fc6800b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 -- 172.21.15.181:0/1635735198 shutdown_connections 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.153+0000 7fc687814640 1 -- 172.21.15.181:0/1635735198 wait complete. 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.154+0000 7fc687814640 1 Processor -- start 2024-03-29T04:09:51.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.154+0000 7fc687814640 1 -- start start 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc680071a60 0x7fc6800b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 0x7fc6801e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc68015d0b0 con 0x7fc680160830 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc68015cdd0 con 0x7fc680145c60 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6800cad70 con 0x7fc680071a60 2024-03-29T04:09:51.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50916/0 (socket says 172.21.15.181:50916) 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 -- 172.21.15.181:0/1668411274 learned_addr learned my addr 172.21.15.181:0/1668411274 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc684daf640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 0x7fc6801e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc6855b0640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc680071a60 0x7fc6800b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 -- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc680071a60 msgr2=0x7fc6800b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc680071a60 0x7fc6800b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 -- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 msgr2=0x7fc6801e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 0x7fc6801e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 -- 172.21.15.181:0/1668411274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6801e6630 con 0x7fc680160830 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc684daf640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 0x7fc6801e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc685db1640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc67800efc0 tx=0x7fc67800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678018020 con 0x7fc680160830 2024-03-29T04:09:51.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678014480 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6801e6910 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.155+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678014870 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.156+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6801e6ed0 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.157+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc680075a10 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.157+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc678005d10 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.158+0000 7fc675ffb640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 0x7fc66406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.158+0000 7fc6855b0640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 0x7fc66406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.158+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc67808c330 con 0x7fc680160830 2024-03-29T04:09:51.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.158+0000 7fc6855b0640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 0x7fc66406ea20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc67c036000 tx=0x7fc67c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.162+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc678022070 con 0x7fc680160830 2024-03-29T04:09:51.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.338+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6800ca610 con 0x7fc680160830 2024-03-29T04:09:51.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.339+0000 7fc675ffb640 1 -- 172.21.15.181:0/1668411274 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7fc678010040 con 0x7fc680160830 2024-03-29T04:09:51.318 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:51.319 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 msgr2=0x7fc66406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 0x7fc66406ea20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc67c036000 tx=0x7fc67c03c040 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 msgr2=0x7fc6800b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc67800efc0 tx=0x7fc67800c490 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 shutdown_connections 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc680160830 0x7fc6800b2ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc680145c60 0x7fc6801e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc66406c560 0x7fc66406ea20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 --2- 172.21.15.181:0/1668411274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc680071a60 0x7fc6800b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 >> 172.21.15.181:0/1668411274 conn(0x7fc6800b1c90 msgr2=0x7fc6800b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 shutdown_connections 2024-03-29T04:09:51.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.341+0000 7fc687814640 1 -- 172.21.15.181:0/1668411274 wait complete. 2024-03-29T04:09:51.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:51.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.457+0000 7fe1db900640 1 Processor -- start 2024-03-29T04:09:51.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 -- start start 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe1d40ba090 0x7fe1d40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d406c620 0x7fe1d406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40b66e0 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d40d2ff0 con 0x7fe1d40ba090 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.458+0000 7fe1db900640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d40d32d0 con 0x7fe1d406c620 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50918/0 (socket says 172.21.15.181:50918) 2024-03-29T04:09:51.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 -- 172.21.15.181:0/3867985729 learned_addr learned my addr 172.21.15.181:0/3867985729 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d8e9b640 1 --2- 172.21.15.181:0/3867985729 >> v2:172.21.15.187:3300/0 conn(0x7fe1d40ba090 0x7fe1d40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d9e9d640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d406c620 0x7fe1d406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 -- 172.21.15.181:0/3867985729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d406c620 msgr2=0x7fe1d406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d406c620 0x7fe1d406ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 -- 172.21.15.181:0/3867985729 >> v2:172.21.15.187:3300/0 conn(0x7fe1d40ba090 msgr2=0x7fe1d40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3867985729 >> v2:172.21.15.187:3300/0 conn(0x7fe1d40ba090 0x7fe1d40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.459+0000 7fe1d969c640 1 -- 172.21.15.181:0/3867985729 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d40d35b0 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b7240 tx=0x7fe1c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83661eb7bd3f19de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1cbfff640 1 -- 172.21.15.181:0/3867985729 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4035020 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1cbfff640 1 -- 172.21.15.181:0/3867985729 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4031870 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1cbfff640 1 -- 172.21.15.181:0/3867985729 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4031c40 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 -- 172.21.15.181:0/3867985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40b5dc0 msgr2=0x7fe1d40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b7240 tx=0x7fe1c402b910 comp rx=0 tx=0).stop 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 -- 172.21.15.181:0/3867985729 shutdown_connections 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d406c620 0x7fe1d406ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 --2- 172.21.15.181:0/3867985729 >> v2:172.21.15.187:3300/0 conn(0x7fe1d40ba090 0x7fe1d40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 --2- 172.21.15.181:0/3867985729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40b5dc0 0x7fe1d40b61a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 -- 172.21.15.181:0/3867985729 >> 172.21.15.181:0/3867985729 conn(0x7fe1d40678f0 msgr2=0x7fe1d40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 -- 172.21.15.181:0/3867985729 shutdown_connections 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.460+0000 7fe1db900640 1 -- 172.21.15.181:0/3867985729 wait complete. 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.461+0000 7fe1db900640 1 Processor -- start 2024-03-29T04:09:51.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.461+0000 7fe1db900640 1 -- start start 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d406c620 0x7fe1d41e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d40b5dc0 0x7fe1d41f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d40d4090 con 0x7fe1d40ba090 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d40d3b70 con 0x7fe1d406c620 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1db900640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d415e880 con 0x7fe1d40b5dc0 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1d9e9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1d9e9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50922/0 (socket says 172.21.15.181:50922) 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1d9e9d640 1 -- 172.21.15.181:0/3592215408 learned_addr learned my addr 172.21.15.181:0/3592215408 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d406c620 0x7fe1d41e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.462+0000 7fe1d8e9b640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d40b5dc0 0x7fe1d41f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 -- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d40b5dc0 msgr2=0x7fe1d41f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d40b5dc0 0x7fe1d41f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 -- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d406c620 msgr2=0x7fe1d41e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d406c620 0x7fe1d41e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 -- 172.21.15.181:0/3592215408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d415fd30 con 0x7fe1d40ba090 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1d9e9d640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00eed0 tx=0x7fe1cc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc018020 con 0x7fe1d40ba090 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc014400 con 0x7fe1d40ba090 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d4160010 con 0x7fe1d40ba090 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0147f0 con 0x7fe1d40ba090 2024-03-29T04:09:51.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.463+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d41608b0 con 0x7fe1d40ba090 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.464+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d4160ab0 con 0x7fe1d40ba090 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.465+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe1cc01b020 con 0x7fe1d40ba090 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.465+0000 7fe1c9ffb640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 0x7fe1b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.465+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1cc08bfc0 con 0x7fe1d40ba090 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.465+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 0x7fe1b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.466+0000 7fe1d969c640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 0x7fe1b806e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4036000 tx=0x7fe1c403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.470+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe1cc022030 con 0x7fe1d40ba090 2024-03-29T04:09:51.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.644+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1d4075890 con 0x7fe1d40ba090 2024-03-29T04:09:51.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.645+0000 7fe1c9ffb640 1 -- 172.21.15.181:0/3592215408 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe1cc059ce0 con 0x7fe1d40ba090 2024-03-29T04:09:51.624 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:51.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 msgr2=0x7fe1b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 0x7fe1b806e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4036000 tx=0x7fe1c403c040 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 msgr2=0x7fe1d4160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00eed0 tx=0x7fe1cc00b480 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 shutdown_connections 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1d40ba090 0x7fe1d4160dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe1b806c530 0x7fe1b806e9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1d40b5dc0 0x7fe1d41f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 --2- 172.21.15.181:0/3592215408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1d406c620 0x7fe1d41e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 >> 172.21.15.181:0/3592215408 conn(0x7fe1d40678f0 msgr2=0x7fe1d40b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 shutdown_connections 2024-03-29T04:09:51.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.647+0000 7fe1db900640 1 -- 172.21.15.181:0/3592215408 wait complete. 2024-03-29T04:09:51.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:51.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.761+0000 7fd875d15640 1 Processor -- start 2024-03-29T04:09:51.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.761+0000 7fd875d15640 1 -- start start 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd875d15640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd870144cb0 0x7fd8700b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd875d15640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd870072d50 0x7fd8700ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd875d15640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8700b35e0 con 0x7fd87015f800 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd875d15640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd8700b3e80 con 0x7fd870144cb0 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd875d15640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd8700b38c0 con 0x7fd870072d50 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd870144cb0 0x7fd8700b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd870144cb0 0x7fd8700b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50926/0 (socket says 172.21.15.181:50926) 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 -- 172.21.15.181:0/368185577 learned_addr learned my addr 172.21.15.181:0/368185577 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86ffff640 1 --2- 172.21.15.181:0/368185577 >> v2:172.21.15.187:3300/0 conn(0x7fd870072d50 0x7fd8700ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 -- 172.21.15.181:0/368185577 >> v2:172.21.15.187:3300/0 conn(0x7fd870072d50 msgr2=0x7fd8700ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 --2- 172.21.15.181:0/368185577 >> v2:172.21.15.187:3300/0 conn(0x7fd870072d50 0x7fd8700ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 -- 172.21.15.181:0/368185577 >> v1:172.21.15.187:6789/0 conn(0x7fd87015f800 legacy=0x7fd870072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.762+0000 7fd86f7fe640 1 -- 172.21.15.181:0/368185577 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8700b3ba0 con 0x7fd870144cb0 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.763+0000 7fd86f7fe640 1 --2- 172.21.15.181:0/368185577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8700b2f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd87006c4c0 tx=0x7fd86002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b267a6119bf3b9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.763+0000 7fd86e7fc640 1 -- 172.21.15.181:0/368185577 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860034070 con 0x7fd870144cb0 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.763+0000 7fd86e7fc640 1 -- 172.21.15.181:0/368185577 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd860030860 con 0x7fd870144cb0 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.763+0000 7fd86e7fc640 1 -- 172.21.15.181:0/368185577 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860030c30 con 0x7fd870144cb0 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 -- 172.21.15.181:0/368185577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 msgr2=0x7fd8700b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 --2- 172.21.15.181:0/368185577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8700b2f20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd87006c4c0 tx=0x7fd86002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 -- 172.21.15.181:0/368185577 shutdown_connections 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 --2- 172.21.15.181:0/368185577 >> v2:172.21.15.187:3300/0 conn(0x7fd870072d50 0x7fd8700ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 --2- 172.21.15.181:0/368185577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8700b2f20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 -- 172.21.15.181:0/368185577 >> 172.21.15.181:0/368185577 conn(0x7fd8700b5b00 msgr2=0x7fd87006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 -- 172.21.15.181:0/368185577 shutdown_connections 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.764+0000 7fd875d15640 1 -- 172.21.15.181:0/368185577 wait complete. 2024-03-29T04:09:51.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.765+0000 7fd875d15640 1 Processor -- start 2024-03-29T04:09:51.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.765+0000 7fd875d15640 1 -- start start 2024-03-29T04:09:51.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd870072d50 0x7fd8701f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd87015f800 0x7fd8700cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8700ce590 con 0x7fd870144cb0 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd87015fc40 con 0x7fd87015f800 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd875d15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd8700ce710 con 0x7fd870072d50 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50936/0 (socket says 172.21.15.181:50936) 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 -- 172.21.15.181:0/54628929 learned_addr learned my addr 172.21.15.181:0/54628929 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86ffff640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd87015f800 0x7fd8700cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86f7fe640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd870072d50 0x7fd8701f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 -- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd870072d50 msgr2=0x7fd8701f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd870072d50 0x7fd8701f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 -- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd87015f800 msgr2=0x7fd8700cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd87015f800 0x7fd8700cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.766+0000 7fd86effd640 1 -- 172.21.15.181:0/54628929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8700ca4b0 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd86effd640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c400 tx=0x7fd86400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864017070 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8640052d0 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8700ca790 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864003830 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.767+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8700cad50 con 0x7fd870144cb0 2024-03-29T04:09:51.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.768+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd870075890 con 0x7fd870144cb0 2024-03-29T04:09:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.768+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd86401a070 con 0x7fd870144cb0 2024-03-29T04:09:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.768+0000 7fd857fff640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 0x7fd85c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.769+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd86408ace0 con 0x7fd870144cb0 2024-03-29T04:09:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.769+0000 7fd86f7fe640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 0x7fd85c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:51.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.769+0000 7fd86f7fe640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 0x7fd85c06e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd860036000 tx=0x7fd860008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:51.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.773+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd864058970 con 0x7fd870144cb0 2024-03-29T04:09:51.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.946+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8700b4030 con 0x7fd870144cb0 2024-03-29T04:09:51.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.947+0000 7fd857fff640 1 -- 172.21.15.181:0/54628929 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fd86405cc70 con 0x7fd870144cb0 2024-03-29T04:09:51.927 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 msgr2=0x7fd85c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 0x7fd85c06e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd860036000 tx=0x7fd860008030 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 msgr2=0x7fd8701f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c400 tx=0x7fd86400c8c0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 shutdown_connections 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd87015f800 0x7fd8700cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd870144cb0 0x7fd8701f4900 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd85c06c530 0x7fd85c06e9f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 --2- 172.21.15.181:0/54628929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd870072d50 0x7fd8701f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:51.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 >> 172.21.15.181:0/54628929 conn(0x7fd8700b5b00 msgr2=0x7fd8700d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 shutdown_connections 2024-03-29T04:09:51.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:51.949+0000 7fd875d15640 1 -- 172.21.15.181:0/54628929 wait complete. 2024-03-29T04:09:51.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:52.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.063+0000 7f364cf32640 1 Processor -- start 2024-03-29T04:09:52.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.063+0000 7f364cf32640 1 -- start start 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364cf32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648072d50 0x7f36480b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364cf32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f364815f800 0x7f36480ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364cf32640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364815fcb0 con 0x7f3648144cb0 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364cf32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f36480b3e80 con 0x7f364815f800 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364cf32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36480b35e0 con 0x7f3648072d50 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f3646d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f364815f800 0x7f36480ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f3646d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f364815f800 0x7f36480ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50946/0 (socket says 172.21.15.181:50946) 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f3646d9d640 1 -- 172.21.15.181:0/2663806848 learned_addr learned my addr 172.21.15.181:0/2663806848 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f364659c640 1 --1- 172.21.15.181:0/2663806848 >> v1:172.21.15.187:6789/0 conn(0x7f3648144cb0 0x7f3648072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59390/0 (socket says 172.21.15.181:59390) 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.064+0000 7f3645d9b640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648072d50 0x7f36480b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f3646d9d640 1 -- 172.21.15.181:0/2663806848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648072d50 msgr2=0x7f36480b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f3646d9d640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648072d50 0x7f36480b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f364559a640 1 -- 172.21.15.181:0/2663806848 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218065834 0 0) 0x7f3638001490 con 0x7f3648144cb0 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f3646d9d640 1 -- 172.21.15.181:0/2663806848 >> v1:172.21.15.187:6789/0 conn(0x7f3648144cb0 legacy=0x7f3648072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f3646d9d640 1 -- 172.21.15.181:0/2663806848 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36480b38c0 con 0x7f364815f800 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f3646d9d640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f364815f800 0x7f36480ce050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3640009870 tx=0x7f364002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e5fef48805084 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f364559a640 1 -- 172.21.15.181:0/2663806848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640034070 con 0x7f364815f800 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f364559a640 1 -- 172.21.15.181:0/2663806848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3640030840 con 0x7f364815f800 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.065+0000 7f364559a640 1 -- 172.21.15.181:0/2663806848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640030c10 con 0x7f364815f800 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 -- 172.21.15.181:0/2663806848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f364815f800 msgr2=0x7f36480ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f364815f800 0x7f36480ce050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3640009870 tx=0x7f364002b980 comp rx=0 tx=0).stop 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 -- 172.21.15.181:0/2663806848 shutdown_connections 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f364815f800 0x7f36480ce050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 --2- 172.21.15.181:0/2663806848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648072d50 0x7f36480b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 -- 172.21.15.181:0/2663806848 >> 172.21.15.181:0/2663806848 conn(0x7f36480b5b00 msgr2=0x7f364806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 -- 172.21.15.181:0/2663806848 shutdown_connections 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.066+0000 7f364cf32640 1 -- 172.21.15.181:0/2663806848 wait complete. 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.067+0000 7f364cf32640 1 Processor -- start 2024-03-29T04:09:52.047 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.067+0000 7f364cf32640 1 -- start start 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648144cb0 0x7f36480ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364815f800 0x7f364813e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36480ce590 con 0x7f3648072d50 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36480cdb30 con 0x7f364815f800 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364cf32640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36480ce710 con 0x7f3648144cb0 2024-03-29T04:09:52.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50948/0 (socket says 172.21.15.181:50948) 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 -- 172.21.15.181:0/2602695946 learned_addr learned my addr 172.21.15.181:0/2602695946 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f3646d9d640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364815f800 0x7f364813e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f3645d9b640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648144cb0 0x7f36480ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 -- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648144cb0 msgr2=0x7f36480ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648144cb0 0x7f36480ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 -- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364815f800 msgr2=0x7f364813e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364815f800 0x7f364813e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.068+0000 7f364659c640 1 -- 172.21.15.181:0/2602695946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364814ba80 con 0x7f3648072d50 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f364659c640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f363800c850 tx=0x7f363800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638018070 con 0x7f3648072d50 2024-03-29T04:09:52.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363800a640 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364814bd40 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800aa70 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.069+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364814c300 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.070+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3648075890 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.070+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f363801b070 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.071+0000 7f3636ffd640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 0x7f362806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.071+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f363808afb0 con 0x7f3648072d50 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.071+0000 7f3645d9b640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 0x7f362806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.071+0000 7f3645d9b640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 0x7f362806ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36480d0c00 tx=0x7f363c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.075+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f364815fcb0 con 0x7f3648072d50 2024-03-29T04:09:52.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.250+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f36480b4030 con 0x7f3648072d50 2024-03-29T04:09:52.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.250+0000 7f3636ffd640 1 -- 172.21.15.181:0/2602695946 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3638058c40 con 0x7f3648072d50 2024-03-29T04:09:52.230 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 msgr2=0x7f362806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 0x7f362806ea20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36480d0c00 tx=0x7f363c009190 comp rx=0 tx=0).stop 2024-03-29T04:09:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 msgr2=0x7f364815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f363800c850 tx=0x7f363800cd10 comp rx=0 tx=0).stop 2024-03-29T04:09:52.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 shutdown_connections 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f364815f800 0x7f364813e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f362806c560 0x7f362806ea20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3648144cb0 0x7f36480ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 --2- 172.21.15.181:0/2602695946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3648072d50 0x7f364815d9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.252+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 >> 172.21.15.181:0/2602695946 conn(0x7f36480b5b00 msgr2=0x7f3648072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.253+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 shutdown_connections 2024-03-29T04:09:52.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.253+0000 7f364cf32640 1 -- 172.21.15.181:0/2602695946 wait complete. 2024-03-29T04:09:52.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:52.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.369+0000 7f7ba643c640 1 Processor -- start 2024-03-29T04:09:52.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.369+0000 7f7ba643c640 1 -- start start 2024-03-29T04:09:52.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba643c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ba015e5b0 0x7f7ba0162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba643c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 0x7f7ba006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba643c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba01630b0 con 0x7f7ba0073310 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba643c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba0073880 con 0x7f7ba015e5b0 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba643c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba00d0e20 con 0x7f7ba006bc90 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ba015e5b0 0x7f7ba0162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9f7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7ba0073310 0x7f7ba006b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36984/0 (socket says 172.21.15.181:36984) 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9f7fe640 1 -- 172.21.15.181:0/1419256530 learned_addr learned my addr 172.21.15.181:0/1419256530 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba49d9640 1 --2- 172.21.15.181:0/1419256530 >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 0x7f7ba006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9ffff640 1 -- 172.21.15.181:0/1419256530 >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 msgr2=0x7f7ba006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9ffff640 1 --2- 172.21.15.181:0/1419256530 >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 0x7f7ba006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9ffff640 1 -- 172.21.15.181:0/1419256530 >> v1:172.21.15.181:6789/0 conn(0x7f7ba0073310 legacy=0x7f7ba006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9effd640 1 -- 172.21.15.181:0/1419256530 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331730119 0 0) 0x7f7ba01630b0 con 0x7f7ba0073310 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7b9ffff640 1 -- 172.21.15.181:0/1419256530 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba00d1100 con 0x7f7ba015e5b0 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.370+0000 7f7ba49d9640 1 --2- 172.21.15.181:0/1419256530 >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 0x7f7ba006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:52.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.371+0000 7f7b9ffff640 1 --2- 172.21.15.181:0/1419256530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0162b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ba006da30 tx=0x7f7b9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39be52c99561c946 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.371+0000 7f7b9effd640 1 -- 172.21.15.181:0/1419256530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900315a0 con 0x7f7ba015e5b0 2024-03-29T04:09:52.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.371+0000 7f7b9effd640 1 -- 172.21.15.181:0/1419256530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b90037070 con 0x7f7ba015e5b0 2024-03-29T04:09:52.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.371+0000 7f7b9effd640 1 -- 172.21.15.181:0/1419256530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90031960 con 0x7f7ba015e5b0 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 -- 172.21.15.181:0/1419256530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 msgr2=0x7f7ba0162b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 --2- 172.21.15.181:0/1419256530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0162b70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ba006da30 tx=0x7f7b9002b840 comp rx=0 tx=0).stop 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 -- 172.21.15.181:0/1419256530 shutdown_connections 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 --2- 172.21.15.181:0/1419256530 >> v2:172.21.15.187:3300/0 conn(0x7f7ba006bc90 0x7f7ba006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 --2- 172.21.15.181:0/1419256530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0162b70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 -- 172.21.15.181:0/1419256530 >> 172.21.15.181:0/1419256530 conn(0x7f7ba006e720 msgr2=0x7f7ba006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 -- 172.21.15.181:0/1419256530 shutdown_connections 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.372+0000 7f7ba643c640 1 -- 172.21.15.181:0/1419256530 wait complete. 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.373+0000 7f7ba643c640 1 Processor -- start 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.373+0000 7f7ba643c640 1 -- start start 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba006bc90 0x7f7ba00cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0073310 0x7f7ba0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba015e990 con 0x7f7ba015e5b0 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00d16c0 con 0x7f7ba0073310 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba00d1be0 con 0x7f7ba006bc90 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50972/0 (socket says 172.21.15.181:50972) 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 -- 172.21.15.181:0/3598694925 learned_addr learned my addr 172.21.15.181:0/3598694925 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7b9f7fe640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0073310 0x7f7ba0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7b9ffff640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba006bc90 0x7f7ba00cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 -- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba006bc90 msgr2=0x7f7ba00cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba006bc90 0x7f7ba00cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 -- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0073310 msgr2=0x7f7ba0160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0073310 0x7f7ba0160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 -- 172.21.15.181:0/3598694925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba0160190 con 0x7f7ba015e5b0 2024-03-29T04:09:52.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba49d9640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009ea0 tx=0x7f7b98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98018020 con 0x7f7ba015e5b0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b98014480 con 0x7f7ba015e5b0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba00ce250 con 0x7f7ba015e5b0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98014870 con 0x7f7ba015e5b0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.374+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba0161280 con 0x7f7ba015e5b0 2024-03-29T04:09:52.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.375+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0075890 con 0x7f7ba015e5b0 2024-03-29T04:09:52.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.376+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b98005d10 con 0x7f7ba015e5b0 2024-03-29T04:09:52.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.376+0000 7f7b9cff9640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 0x7f7b8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.376+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b9808c660 con 0x7f7ba015e5b0 2024-03-29T04:09:52.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.376+0000 7f7b9ffff640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 0x7f7b8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.377+0000 7f7b9ffff640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 0x7f7b8c06e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b90036000 tx=0x7f7b9003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.381+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b98022030 con 0x7f7ba015e5b0 2024-03-29T04:09:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.555+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7ba0072be0 con 0x7f7ba015e5b0 2024-03-29T04:09:52.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.555+0000 7f7b9cff9640 1 -- 172.21.15.181:0/3598694925 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f7b98010040 con 0x7f7ba015e5b0 2024-03-29T04:09:52.535 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:52.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 msgr2=0x7f7b8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 0x7f7b8c06e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b90036000 tx=0x7f7b9003d040 comp rx=0 tx=0).stop 2024-03-29T04:09:52.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 msgr2=0x7f7ba0146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009ea0 tx=0x7f7b98009f90 comp rx=0 tx=0).stop 2024-03-29T04:09:52.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 shutdown_connections 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba015e5b0 0x7f7ba0146510 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0073310 0x7f7ba0160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7b8c06c530 0x7f7b8c06e9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 --2- 172.21.15.181:0/3598694925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba006bc90 0x7f7ba00cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 >> 172.21.15.181:0/3598694925 conn(0x7f7ba006e720 msgr2=0x7f7ba0070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 shutdown_connections 2024-03-29T04:09:52.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.557+0000 7f7ba643c640 1 -- 172.21.15.181:0/3598694925 wait complete. 2024-03-29T04:09:52.548 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.669+0000 7f0176aa5640 1 Processor -- start 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.669+0000 7f0176aa5640 1 -- start start 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0176aa5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0170072610 0x7f017015f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0176aa5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01700ce840 con 0x7f0170072230 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0176aa5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01700ceb20 con 0x7f017006db10 2024-03-29T04:09:52.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0176aa5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0170158f10 con 0x7f0170072610 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0174841640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0170072230 0x7f017006bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36994/0 (socket says 172.21.15.181:36994) 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f0174841640 1 -- 172.21.15.181:0/1398458382 learned_addr learned my addr 172.21.15.181:0/1398458382 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f016ffff640 1 --2- 172.21.15.181:0/1398458382 >> v2:172.21.15.187:3300/0 conn(0x7f0170072610 0x7f017015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730763998 0 0) 0x7f01700ce840 con 0x7f0170072230 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.670+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0158003520 con 0x7f0170072230 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016ffff640 1 -- 172.21.15.181:0/1398458382 >> v1:172.21.15.187:6789/0 conn(0x7f017006db10 legacy=0x7f017006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 279207995 0 0) 0x7f01700ceb20 con 0x7f017006db10 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016ffff640 1 -- 172.21.15.181:0/1398458382 >> v1:172.21.15.181:6789/0 conn(0x7f0170072230 legacy=0x7f017006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016ffff640 1 -- 172.21.15.181:0/1398458382 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017015ec00 con 0x7f0170072610 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016ffff640 1 --2- 172.21.15.181:0/1398458382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f017015f400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01640097a0 tx=0x7f016402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c7ef350f36a1bc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.671+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164034070 con 0x7f0170072610 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164030610 con 0x7f0170072610 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f016f7fe640 1 -- 172.21.15.181:0/1398458382 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640309e0 con 0x7f0170072610 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 -- 172.21.15.181:0/1398458382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 msgr2=0x7f017015f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 --2- 172.21.15.181:0/1398458382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f017015f400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01640097a0 tx=0x7f016402b960 comp rx=0 tx=0).stop 2024-03-29T04:09:52.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 -- 172.21.15.181:0/1398458382 shutdown_connections 2024-03-29T04:09:52.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 --2- 172.21.15.181:0/1398458382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f017015f400 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 -- 172.21.15.181:0/1398458382 >> 172.21.15.181:0/1398458382 conn(0x7f01700d0cb0 msgr2=0x7f01700d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 -- 172.21.15.181:0/1398458382 shutdown_connections 2024-03-29T04:09:52.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.672+0000 7f0176aa5640 1 -- 172.21.15.181:0/1398458382 wait complete. 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.673+0000 7f0176aa5640 1 Processor -- start 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.673+0000 7f0176aa5640 1 -- start start 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f017006db10 0x7f01701570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f01700aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f017006e630 con 0x7f0170072230 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f017015ef70 con 0x7f0170072610 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0176aa5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0170072ac0 con 0x7f017006db10 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f016ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f016ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50986/0 (socket says 172.21.15.181:50986) 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f016ffff640 1 -- 172.21.15.181:0/2138456297 learned_addr learned my addr 172.21.15.181:0/2138456297 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0175042640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f01700aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.674+0000 7f0174841640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f017006db10 0x7f01701570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 -- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f017006db10 msgr2=0x7f01701570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f017006db10 0x7f01701570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 -- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 msgr2=0x7f01700aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f01700aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 -- 172.21.15.181:0/2138456297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01701f3f90 con 0x7f0170072230 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016ffff640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01640098c0 tx=0x7f01640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164049070 con 0x7f0170072230 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f016403a890 con 0x7f0170072230 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01701f4270 con 0x7f0170072230 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164034040 con 0x7f0170072230 2024-03-29T04:09:52.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.675+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01701f4830 con 0x7f0170072230 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.676+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0170075a10 con 0x7f0170072230 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.676+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f016404c070 con 0x7f0170072230 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.677+0000 7f016d7fa640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 0x7f015c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.677+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01640ad600 con 0x7f0170072230 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.677+0000 7f0174841640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 0x7f015c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.657 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.677+0000 7f0174841640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 0x7f015c06e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0160005880 tx=0x7f0160009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.661 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.681+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f016403f070 con 0x7f0170072230 2024-03-29T04:09:52.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.856+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01700b1a10 con 0x7f0170072230 2024-03-29T04:09:52.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.857+0000 7f016d7fa640 1 -- 172.21.15.181:0/2138456297 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f016407b320 con 0x7f0170072230 2024-03-29T04:09:52.837 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:52.837 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.859+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 msgr2=0x7f015c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.859+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 0x7f015c06e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0160005880 tx=0x7f0160009040 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.859+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 msgr2=0x7f01700ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.859+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f01640098c0 tx=0x7f01640046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.859+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 shutdown_connections 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0170072610 0x7f01700aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0170072230 0x7f01700ae8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f015c06c530 0x7f015c06e9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 --2- 172.21.15.181:0/2138456297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f017006db10 0x7f01701570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 >> 172.21.15.181:0/2138456297 conn(0x7f01700d0cb0 msgr2=0x7f01700d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 shutdown_connections 2024-03-29T04:09:52.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.860+0000 7f0176aa5640 1 -- 172.21.15.181:0/2138456297 wait complete. 2024-03-29T04:09:52.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:52.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:52.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 Processor -- start 2024-03-29T04:09:52.955 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 -- start start 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9ff0144cb0 0x7f9ff00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 0x7f9ff00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff00b35e0 con 0x7f9ff015f800 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff00b3e80 con 0x7f9ff0144cb0 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.976+0000 7f9ff4ffd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff00b38c0 con 0x7f9ff0072d50 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9ff0144cb0 0x7f9ff00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9feed9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 0x7f9ff00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fe7fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9ff015f800 0x7f9ff0072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59404/0 (socket says 172.21.15.181:59404) 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9ff0144cb0 0x7f9ff00b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:59774/0 (socket says 172.21.15.181:59774) 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fe7fff640 1 -- 172.21.15.181:0/3006116333 learned_addr learned my addr 172.21.15.181:0/3006116333 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 -- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 msgr2=0x7f9ff00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:52.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 0x7f9ff00ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 -- 172.21.15.181:0/3006116333 >> v1:172.21.15.187:6789/0 conn(0x7f9ff015f800 legacy=0x7f9ff0072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9fee59c640 1 -- 172.21.15.181:0/3006116333 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00b3ba0 con 0x7f9ff0144cb0 2024-03-29T04:09:52.958 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.977+0000 7f9feed9d640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 0x7f9ff00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9fee59c640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0144cb0 0x7f9ff00b2f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba40 tx=0x7f9fe002ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b0e6578906027 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9fedd9b640 1 -- 172.21.15.181:0/3006116333 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe003b070 con 0x7f9ff0144cb0 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9fedd9b640 1 -- 172.21.15.181:0/3006116333 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe002ec00 con 0x7f9ff0144cb0 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9fedd9b640 1 -- 172.21.15.181:0/3006116333 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0036440 con 0x7f9ff0144cb0 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0144cb0 msgr2=0x7f9ff00b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.959 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0144cb0 0x7f9ff00b2f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba40 tx=0x7f9fe002ded0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/3006116333 shutdown_connections 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0072d50 0x7f9ff00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/3006116333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0144cb0 0x7f9ff00b2f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.978+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/3006116333 >> 172.21.15.181:0/3006116333 conn(0x7f9ff00b5b00 msgr2=0x7f9ff006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.979+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/3006116333 shutdown_connections 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.979+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/3006116333 wait complete. 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 Processor -- start 2024-03-29T04:09:52.960 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 -- start start 2024-03-29T04:09:52.962 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 0x7f9ff00ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00ce590 con 0x7f9ff015f800 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff015fc40 con 0x7f9ff0072d50 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.980+0000 7f9ff4ffd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff00ce710 con 0x7f9ff0144cb0 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9feed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fee59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 0x7f9ff00ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9feed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50988/0 (socket says 172.21.15.181:50988) 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9feed9d640 1 -- 172.21.15.181:0/183396889 learned_addr learned my addr 172.21.15.181:0/183396889 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:47380/0 (socket says 172.21.15.181:47380) 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 -- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 msgr2=0x7f9ff00ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 0x7f9ff00ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 -- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 msgr2=0x7f9ff015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 -- 172.21.15.181:0/183396889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00af930 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fee59c640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 0x7f9ff00ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9feed9d640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe7fff640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c500 tx=0x7f9fd800c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8018070 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd8007560 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80049e0 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff00cbdd0 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.981+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff00ad940 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.982+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff0075890 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.983+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9fd8004b80 con 0x7f9ff0144cb0 2024-03-29T04:09:52.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.983+0000 7f9fe6ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 0x7f9fd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.983+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9fd808adb0 con 0x7f9ff0144cb0 2024-03-29T04:09:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.983+0000 7f9fee59c640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 0x7f9fd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:52.965 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.984+0000 7f9fee59c640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 0x7f9fd006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0038000 tx=0x7f9fe0032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:52.967 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:52.989+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9fd804e8e0 con 0x7f9ff0144cb0 2024-03-29T04:09:53.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.162+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ff00b4030 con 0x7f9ff0144cb0 2024-03-29T04:09:53.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.162+0000 7f9fe6ffd640 1 -- 172.21.15.181:0/183396889 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9fd80589f0 con 0x7f9ff0144cb0 2024-03-29T04:09:53.142 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:53.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 msgr2=0x7f9fd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 0x7f9fd006e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0038000 tx=0x7f9fe0032000 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 msgr2=0x7f9ff00af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c500 tx=0x7f9fd800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 shutdown_connections 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ff015f800 0x7f9ff015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9fd006c530 0x7f9fd006e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ff0144cb0 0x7f9ff00af3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 --2- 172.21.15.181:0/183396889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ff0072d50 0x7f9ff00ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 >> 172.21.15.181:0/183396889 conn(0x7f9ff00b5b00 msgr2=0x7f9ff0071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 shutdown_connections 2024-03-29T04:09:53.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.164+0000 7f9ff4ffd640 1 -- 172.21.15.181:0/183396889 wait complete. 2024-03-29T04:09:53.155 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.279+0000 7fe29f185640 1 Processor -- start 2024-03-29T04:09:53.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.279+0000 7fe29f185640 1 -- start start 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe29815f800 0x7fe298072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe2980b3e80 con 0x7fe298144cb0 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2980b35e0 con 0x7fe29815f800 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29f185640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2980b38c0 con 0x7fe298072b80 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29cf21640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29cf21640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50996/0 (socket says 172.21.15.181:50996) 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 --2- 172.21.15.181:0/2819269795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29cf21640 1 -- 172.21.15.181:0/2819269795 learned_addr learned my addr 172.21.15.181:0/2819269795 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe297fff640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.187:3300/0 conn(0x7fe29815f800 0x7fe298072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 -- 172.21.15.181:0/2819269795 >> v2:172.21.15.187:3300/0 conn(0x7fe29815f800 msgr2=0x7fe298072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.187:3300/0 conn(0x7fe29815f800 0x7fe298072640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 -- 172.21.15.181:0/2819269795 >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 msgr2=0x7fe2980b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29d722640 1 -- 172.21.15.181:0/2819269795 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2980b3ba0 con 0x7fe298072b80 2024-03-29T04:09:53.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.280+0000 7fe29cf21640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:53.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.281+0000 7fe29d722640 1 --2- 172.21.15.181:0/2819269795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298071420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe29815ed50 tx=0x7fe29002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af2f03a7b28b3e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.281+0000 7fe2977fe640 1 -- 172.21.15.181:0/2819269795 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290039070 con 0x7fe298072b80 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.281+0000 7fe2977fe640 1 -- 172.21.15.181:0/2819269795 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29002cbd0 con 0x7fe298072b80 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.281+0000 7fe2977fe640 1 -- 172.21.15.181:0/2819269795 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290034440 con 0x7fe298072b80 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 -- 172.21.15.181:0/2819269795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 msgr2=0x7fe298071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 --2- 172.21.15.181:0/2819269795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298071420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe29815ed50 tx=0x7fe29002be20 comp rx=0 tx=0).stop 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 -- 172.21.15.181:0/2819269795 shutdown_connections 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 --2- 172.21.15.181:0/2819269795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298071420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.187:3300/0 conn(0x7fe29815f800 0x7fe298072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 --2- 172.21.15.181:0/2819269795 >> v2:172.21.15.181:3300/0 conn(0x7fe298144cb0 0x7fe2980b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 -- 172.21.15.181:0/2819269795 >> 172.21.15.181:0/2819269795 conn(0x7fe2980b5b00 msgr2=0x7fe29806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 -- 172.21.15.181:0/2819269795 shutdown_connections 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.282+0000 7fe29f185640 1 -- 172.21.15.181:0/2819269795 wait complete. 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.283+0000 7fe29f185640 1 Processor -- start 2024-03-29T04:09:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.283+0000 7fe29f185640 1 -- start start 2024-03-29T04:09:53.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 0x7fe2981408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe298071960 con 0x7fe29815f800 2024-03-29T04:09:53.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe29815fc40 con 0x7fe298144cb0 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29f185640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe298071ae0 con 0x7fe298072b80 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51002/0 (socket says 172.21.15.181:51002) 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 -- 172.21.15.181:0/487963625 learned_addr learned my addr 172.21.15.181:0/487963625 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29cf21640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe297fff640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 0x7fe2981408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 -- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 msgr2=0x7fe298148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 -- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 msgr2=0x7fe2981408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 0x7fe2981408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 -- 172.21.15.181:0/487963625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298167250 con 0x7fe29815f800 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe297fff640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 0x7fe2981408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:53.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe29d722640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe290009e80 tx=0x7fe290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.284+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290039050 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.285+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe290045070 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.285+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe298167530 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.285+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290009450 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.285+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe298167af0 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.285+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe298075890 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.286+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe29003e850 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.286+0000 7fe2957fa640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 0x7fe27c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.286+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe2900ae6e0 con 0x7fe29815f800 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.286+0000 7fe29cf21640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 0x7fe27c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.287+0000 7fe29cf21640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 0x7fe27c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006ca0 tx=0x7fe28c009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.291+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe29007c520 con 0x7fe29815f800 2024-03-29T04:09:53.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.468+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe29815e800 con 0x7fe29815f800 2024-03-29T04:09:53.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.469+0000 7fe2957fa640 1 -- 172.21.15.181:0/487963625 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fe290080820 con 0x7fe29815f800 2024-03-29T04:09:53.448 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:53.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 msgr2=0x7fe27c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 0x7fe27c06e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006ca0 tx=0x7fe28c009240 comp rx=0 tx=0).stop 2024-03-29T04:09:53.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 msgr2=0x7fe298140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe290009e80 tx=0x7fe290004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 shutdown_connections 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe29815f800 0x7fe298140e10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe298144cb0 0x7fe2981408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe27c06c530 0x7fe27c06e9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 --2- 172.21.15.181:0/487963625 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe298072b80 0x7fe298148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 >> 172.21.15.181:0/487963625 conn(0x7fe2980b5b00 msgr2=0x7fe29806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 shutdown_connections 2024-03-29T04:09:53.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.471+0000 7fe29f185640 1 -- 172.21.15.181:0/487963625 wait complete. 2024-03-29T04:09:53.461 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:53.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.585+0000 7f28c80c8640 1 Processor -- start 2024-03-29T04:09:53.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.585+0000 7f28c80c8640 1 -- start start 2024-03-29T04:09:53.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c80c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c80c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28c00b73d0 0x7f28c00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c80c8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c00b66e0 con 0x7f28c00ba090 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c80c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28c00d2d90 con 0x7f28c00b73d0 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c80c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28c00d2ff0 con 0x7f28c00b5dc0 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c5663640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f28c00ba090 0x7f28c00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36996/0 (socket says 172.21.15.181:36996) 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c5663640 1 -- 172.21.15.181:0/2475069281 learned_addr learned my addr 172.21.15.181:0/2475069281 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c6665640 1 --2- 172.21.15.181:0/2475069281 >> v2:172.21.15.187:3300/0 conn(0x7f28c00b73d0 0x7f28c00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.586+0000 7f28c5e64640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c4e62640 1 -- 172.21.15.181:0/2475069281 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69056731 0 0) 0x7f28c00b66e0 con 0x7f28c00ba090 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c4e62640 1 -- 172.21.15.181:0/2475069281 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28ac003520 con 0x7f28c00ba090 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c6665640 1 -- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 msgr2=0x7f28c00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c6665640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c00b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c6665640 1 -- 172.21.15.181:0/2475069281 >> v1:172.21.15.181:6789/0 conn(0x7f28c00ba090 legacy=0x7f28c00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c6665640 1 -- 172.21.15.181:0/2475069281 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00d32d0 con 0x7f28c00b73d0 2024-03-29T04:09:53.568 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.587+0000 7f28c5e64640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c6665640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00b73d0 0x7f28c00b7880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7204da03712ff0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c4e62640 1 -- 172.21.15.181:0/2475069281 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8035020 con 0x7f28c00b73d0 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c4e62640 1 -- 172.21.15.181:0/2475069281 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8031830 con 0x7f28c00b73d0 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c4e62640 1 -- 172.21.15.181:0/2475069281 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031be0 con 0x7f28c00b73d0 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 -- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00b73d0 msgr2=0x7f28c00b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00b73d0 0x7f28c00b7880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097a0 tx=0x7f28b802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.569 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 -- 172.21.15.181:0/2475069281 shutdown_connections 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00b73d0 0x7f28c00b7880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2475069281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 -- 172.21.15.181:0/2475069281 >> 172.21.15.181:0/2475069281 conn(0x7f28c00678f0 msgr2=0x7f28c00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 -- 172.21.15.181:0/2475069281 shutdown_connections 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.588+0000 7f28c80c8640 1 -- 172.21.15.181:0/2475069281 wait complete. 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.589+0000 7f28c80c8640 1 Processor -- start 2024-03-29T04:09:53.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 -- start start 2024-03-29T04:09:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00ba090 0x7f28c015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28c00d3b70 con 0x7f28c00b73d0 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28c00d3890 con 0x7f28c00ba090 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c80c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28c00d4090 con 0x7f28c00b5dc0 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c5663640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c5663640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51016/0 (socket says 172.21.15.181:51016) 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c5663640 1 -- 172.21.15.181:0/2177169584 learned_addr learned my addr 172.21.15.181:0/2177169584 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c6665640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00ba090 0x7f28c015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.590+0000 7f28c5e64640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 -- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 msgr2=0x7f28c015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 -- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00ba090 msgr2=0x7f28c015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00ba090 0x7f28c015fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 -- 172.21.15.181:0/2177169584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00aaed0 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c5663640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c3e0 tx=0x7f28b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0017070 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0005ce0 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00072d0 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c00ab1b0 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.591+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c00ab770 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.592+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c00d4f70 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.592+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f28b001a070 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.593+0000 7f28b67fc640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 0x7f28a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.593+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f28b008acc0 con 0x7f28c00b73d0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.593+0000 7f28c5e64640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 0x7f28a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.574 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.593+0000 7f28c5e64640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 0x7f28a406e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009bd0 tx=0x7f28bc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.597+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f28c00d4f70 con 0x7f28c00b73d0 2024-03-29T04:09:53.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.771+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28c00ab970 con 0x7f28c00b73d0 2024-03-29T04:09:53.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.772+0000 7f28b67fc640 1 -- 172.21.15.181:0/2177169584 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f28c00ab970 con 0x7f28c00b73d0 2024-03-29T04:09:53.751 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:53.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 msgr2=0x7f28a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 0x7f28a406e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009bd0 tx=0x7f28bc00b040 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 msgr2=0x7f28c00ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c3e0 tx=0x7f28b000c8a0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 shutdown_connections 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28c00ba090 0x7f28c015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28c00b73d0 0x7f28c00ad820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f28a406c530 0x7f28a406e9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 --2- 172.21.15.181:0/2177169584 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28c00b5dc0 0x7f28c015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 >> 172.21.15.181:0/2177169584 conn(0x7f28c00678f0 msgr2=0x7f28c015db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 shutdown_connections 2024-03-29T04:09:53.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.774+0000 7f28c80c8640 1 -- 172.21.15.181:0/2177169584 wait complete. 2024-03-29T04:09:53.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:53.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 Processor -- start 2024-03-29T04:09:53.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 -- start start 2024-03-29T04:09:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe140ba090 0x7fbe140d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140d2ff0 con 0x7fbe140b5dc0 2024-03-29T04:09:53.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140d32d0 con 0x7fbe140b73d0 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe197a8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140d35b0 con 0x7fbe140ba090 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe0bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe140ba090 0x7fbe140d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe12ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe140b5dc0 0x7fbe140b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37004/0 (socket says 172.21.15.181:37004) 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe0bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe140ba090 0x7fbe140d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51024/0 (socket says 172.21.15.181:51024) 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe12ffd640 1 -- 172.21.15.181:0/3852090502 learned_addr learned my addr 172.21.15.181:0/3852090502 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe0bfff640 1 -- 172.21.15.181:0/3852090502 >> v1:172.21.15.187:6789/0 conn(0x7fbe140b73d0 legacy=0x7fbe140b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe0bfff640 1 -- 172.21.15.181:0/3852090502 >> v1:172.21.15.181:6789/0 conn(0x7fbe140b5dc0 legacy=0x7fbe140b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe0bfff640 1 -- 172.21.15.181:0/3852090502 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140d3890 con 0x7fbe140ba090 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.889+0000 7fbe127fc640 1 -- 172.21.15.181:0/3852090502 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800996107 0 0) 0x7fbe140d2ff0 con 0x7fbe140b5dc0 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.890+0000 7fbe0bfff640 1 --2- 172.21.15.181:0/3852090502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140ba090 0x7fbe140d2680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0097d0 tx=0x7fbdfc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c556ea5c05102d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.890+0000 7fbe127fc640 1 -- 172.21.15.181:0/3852090502 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc034070 con 0x7fbe140ba090 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.890+0000 7fbe127fc640 1 -- 172.21.15.181:0/3852090502 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc030610 con 0x7fbe140ba090 2024-03-29T04:09:53.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.890+0000 7fbe127fc640 1 -- 172.21.15.181:0/3852090502 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0309e0 con 0x7fbe140ba090 2024-03-29T04:09:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 -- 172.21.15.181:0/3852090502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140ba090 msgr2=0x7fbe140d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 --2- 172.21.15.181:0/3852090502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140ba090 0x7fbe140d2680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0097d0 tx=0x7fbdfc02b960 comp rx=0 tx=0).stop 2024-03-29T04:09:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 -- 172.21.15.181:0/3852090502 shutdown_connections 2024-03-29T04:09:53.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 --2- 172.21.15.181:0/3852090502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140ba090 0x7fbe140d2680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 -- 172.21.15.181:0/3852090502 >> 172.21.15.181:0/3852090502 conn(0x7fbe140678f0 msgr2=0x7fbe140d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 -- 172.21.15.181:0/3852090502 shutdown_connections 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.891+0000 7fbe197a8640 1 -- 172.21.15.181:0/3852090502 wait complete. 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 Processor -- start 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 -- start start 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe140b73d0 0x7fbe14160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe140ba090 0x7fbe14161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1415e880 con 0x7fbe140b5dc0 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.892+0000 7fbe197a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140d4090 con 0x7fbe140ba090 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe197a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1415ea00 con 0x7fbe140b73d0 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51028/0 (socket says 172.21.15.181:51028) 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 -- 172.21.15.181:0/2191909830 learned_addr learned my addr 172.21.15.181:0/2191909830 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe0bfff640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe140b73d0 0x7fbe14160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe137fe640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe140ba090 0x7fbe14161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 -- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe140b73d0 msgr2=0x7fbe14160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe140b73d0 0x7fbe14160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 -- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe140ba090 msgr2=0x7fbe14161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe140ba090 0x7fbe14161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 -- 172.21.15.181:0/2191909830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140cc190 con 0x7fbe140b5dc0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe12ffd640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c4d0 tx=0x7fbe0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04018070 con 0x7fbe140b5dc0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe040074e0 con 0x7fbe140b5dc0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04004a20 con 0x7fbe140b5dc0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe140cc470 con 0x7fbe140b5dc0 2024-03-29T04:09:53.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.893+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe140cca30 con 0x7fbe140b5dc0 2024-03-29T04:09:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.894+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14075890 con 0x7fbe140b5dc0 2024-03-29T04:09:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.895+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe0401b070 con 0x7fbe140b5dc0 2024-03-29T04:09:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.895+0000 7fbe0b7fe640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 0x7fbdf406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:53.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.895+0000 7fbe0bfff640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 0x7fbdf406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.895+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbe0408acd0 con 0x7fbe140b5dc0 2024-03-29T04:09:53.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.897+0000 7fbe0bfff640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 0x7fbdf406e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc036000 tx=0x7fbdfc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:53.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:53.901+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe04058960 con 0x7fbe140b5dc0 2024-03-29T04:09:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.073+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbe14157420 con 0x7fbe140b5dc0 2024-03-29T04:09:54.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.074+0000 7fbe0b7fe640 1 -- 172.21.15.181:0/2191909830 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fbe0405cc60 con 0x7fbe140b5dc0 2024-03-29T04:09:54.052 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:54.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.075+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 msgr2=0x7fbdf406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.075+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 0x7fbdf406e9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc036000 tx=0x7fbdfc008030 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 msgr2=0x7fbe14155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c4d0 tx=0x7fbe0400c990 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 shutdown_connections 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe140ba090 0x7fbe14161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdf406c530 0x7fbdf406e9f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe140b73d0 0x7fbe14160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 --2- 172.21.15.181:0/2191909830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe140b5dc0 0x7fbe14155a80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 >> 172.21.15.181:0/2191909830 conn(0x7fbe140678f0 msgr2=0x7fbe140b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 shutdown_connections 2024-03-29T04:09:54.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.076+0000 7fbe197a8640 1 -- 172.21.15.181:0/2191909830 wait complete. 2024-03-29T04:09:54.065 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.187+0000 7f12039e2640 1 Processor -- start 2024-03-29T04:09:54.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.187+0000 7f12039e2640 1 -- start start 2024-03-29T04:09:54.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f12039e2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11fc15a960 0x7f11fc071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f12039e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0cb580 0x7f11fc0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f12039e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc0724e0 con 0x7f11fc0cd3b0 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f12039e2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc0730f0 con 0x7f11fc0cb580 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f12039e2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc0727c0 con 0x7f11fc15a960 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f1201f7f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0cb580 0x7f11fc0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f1201f7f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11fc0cb580 0x7f11fc0cddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51032/0 (socket says 172.21.15.181:51032) 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.188+0000 7f1201f7f640 1 -- 172.21.15.181:0/4182762409 learned_addr learned my addr 172.21.15.181:0/4182762409 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f120177e640 1 --2- 172.21.15.181:0/4182762409 >> v2:172.21.15.187:3300/0 conn(0x7f11fc15a960 0x7f11fc071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f1201f7f640 1 -- 172.21.15.181:0/4182762409 >> v2:172.21.15.187:3300/0 conn(0x7f11fc15a960 msgr2=0x7f11fc071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f1201f7f640 1 --2- 172.21.15.181:0/4182762409 >> v2:172.21.15.187:3300/0 conn(0x7f11fc15a960 0x7f11fc071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f1201f7f640 1 -- 172.21.15.181:0/4182762409 >> v1:172.21.15.187:6789/0 conn(0x7f11fc0cd3b0 legacy=0x7f11fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f1201f7f640 1 -- 172.21.15.181:0/4182762409 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc072aa0 con 0x7f11fc0cb580 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.189+0000 7f1201f7f640 1 --2- 172.21.15.181:0/4182762409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0cb580 0x7f11fc0cddd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0691c0 tx=0x7f11f802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cd07b46238b270 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.177 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f11f3fff640 1 -- 172.21.15.181:0/4182762409 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f803b070 con 0x7f11fc0cb580 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f11f3fff640 1 -- 172.21.15.181:0/4182762409 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8036440 con 0x7f11fc0cb580 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f11f3fff640 1 -- 172.21.15.181:0/4182762409 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8036810 con 0x7f11fc0cb580 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 -- 172.21.15.181:0/4182762409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0cb580 msgr2=0x7f11fc0cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 --2- 172.21.15.181:0/4182762409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0cb580 0x7f11fc0cddd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0691c0 tx=0x7f11f802e170 comp rx=0 tx=0).stop 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 -- 172.21.15.181:0/4182762409 shutdown_connections 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 --2- 172.21.15.181:0/4182762409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc0cb580 0x7f11fc0cddd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 --2- 172.21.15.181:0/4182762409 >> v2:172.21.15.187:3300/0 conn(0x7f11fc15a960 0x7f11fc071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 -- 172.21.15.181:0/4182762409 >> 172.21.15.181:0/4182762409 conn(0x7f11fc0b1c90 msgr2=0x7f11fc0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 -- 172.21.15.181:0/4182762409 shutdown_connections 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.190+0000 7f12039e2640 1 -- 172.21.15.181:0/4182762409 wait complete. 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.191+0000 7f12039e2640 1 Processor -- start 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.191+0000 7f12039e2640 1 -- start start 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 0x7f11fc0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11fc0cd3b0 0x7f11fc159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc071fb0 con 0x7f11fc15a960 2024-03-29T04:09:54.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc15ad40 con 0x7f11fc0cd3b0 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f12039e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc072130 con 0x7f11fc0cb580 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51044/0 (socket says 172.21.15.181:51044) 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 -- 172.21.15.181:0/3067639793 learned_addr learned my addr 172.21.15.181:0/3067639793 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f120177e640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 0x7f11fc0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1200f7d640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11fc0cd3b0 0x7f11fc159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 -- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 msgr2=0x7f11fc0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 0x7f11fc0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 -- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11fc0cd3b0 msgr2=0x7f11fc159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11fc0cd3b0 0x7f11fc159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f1201f7f640 1 -- 172.21.15.181:0/3067639793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc0c96d0 con 0x7f11fc15a960 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.192+0000 7f120177e640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 0x7f11fc0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f1201f7f640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11f802e1b0 tx=0x7f11f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f803b040 con 0x7f11fc15a960 2024-03-29T04:09:54.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8046070 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc0c99b0 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80348a0 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.193+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc0c9f70 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.194+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc075a10 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.194+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f11f8034a40 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.195+0000 7f11f1ffb640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 0x7f11e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.195+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f11f80b0400 con 0x7f11fc15a960 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.195+0000 7f120177e640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 0x7f11e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.195+0000 7f120177e640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 0x7f11e006e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0b5280 tx=0x7f11f400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.199+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f11f807e770 con 0x7f11fc15a960 2024-03-29T04:09:54.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.375+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11fc06e2e0 con 0x7f11fc15a960 2024-03-29T04:09:54.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.377+0000 7f11f1ffb640 1 -- 172.21.15.181:0/3067639793 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f11f807e070 con 0x7f11fc15a960 2024-03-29T04:09:54.355 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:54.356 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 msgr2=0x7f11e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 0x7f11e006e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0b5280 tx=0x7f11f400b040 comp rx=0 tx=0).stop 2024-03-29T04:09:54.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 msgr2=0x7f11fc1492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f11f802e1b0 tx=0x7f11f80046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 shutdown_connections 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc15a960 0x7f11fc1492c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11fc0cd3b0 0x7f11fc159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f11e006c530 0x7f11e006e9f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 --2- 172.21.15.181:0/3067639793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11fc0cb580 0x7f11fc0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 >> 172.21.15.181:0/3067639793 conn(0x7f11fc0b1c90 msgr2=0x7f11fc158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 shutdown_connections 2024-03-29T04:09:54.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.379+0000 7f12039e2640 1 -- 172.21.15.181:0/3067639793 wait complete. 2024-03-29T04:09:54.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:54.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:54.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.495+0000 7f2587115640 1 Processor -- start 2024-03-29T04:09:54.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.495+0000 7f2587115640 1 -- start start 2024-03-29T04:09:54.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 0x7f2580162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2580073310 0x7f258006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f258006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f25801630b0 con 0x7f2580073310 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2580073880 con 0x7f258015e5b0 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2587115640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25800d0e20 con 0x7f25800d1be0 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2580073310 0x7f258006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2580073310 0x7f258006b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51060/0 (socket says 172.21.15.181:51060) 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 -- 172.21.15.181:0/2984289736 learned_addr learned my addr 172.21.15.181:0/2984289736 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2584eb1640 1 --2- 172.21.15.181:0/2984289736 >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 0x7f2580162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f25856b2640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f258006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 -- 172.21.15.181:0/2984289736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 msgr2=0x7f258006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f258006bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 -- 172.21.15.181:0/2984289736 >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 msgr2=0x7f2580162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 --2- 172.21.15.181:0/2984289736 >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 0x7f2580162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f257ffff640 1 -- 172.21.15.181:0/2984289736 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25800d1100 con 0x7f2580073310 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.496+0000 7f2584eb1640 1 --2- 172.21.15.181:0/2984289736 >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 0x7f2580162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.497+0000 7f257ffff640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580073310 0x7f258006b580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25800720e0 tx=0x7f256c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d952d13eb307289f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.497+0000 7f257f7fe640 1 -- 172.21.15.181:0/2984289736 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c02e070 con 0x7f2580073310 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.497+0000 7f257f7fe640 1 -- 172.21.15.181:0/2984289736 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c037070 con 0x7f2580073310 2024-03-29T04:09:54.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.497+0000 7f257f7fe640 1 -- 172.21.15.181:0/2984289736 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c031a00 con 0x7f2580073310 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 -- 172.21.15.181:0/2984289736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580073310 msgr2=0x7f258006b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580073310 0x7f258006b580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f25800720e0 tx=0x7f256c02b880 comp rx=0 tx=0).stop 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 -- 172.21.15.181:0/2984289736 shutdown_connections 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f258006bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 --2- 172.21.15.181:0/2984289736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580073310 0x7f258006b580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 --2- 172.21.15.181:0/2984289736 >> v2:172.21.15.187:3300/0 conn(0x7f258015e5b0 0x7f2580162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 -- 172.21.15.181:0/2984289736 >> 172.21.15.181:0/2984289736 conn(0x7f258006e720 msgr2=0x7f258006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 -- 172.21.15.181:0/2984289736 shutdown_connections 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.498+0000 7f2587115640 1 -- 172.21.15.181:0/2984289736 wait complete. 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.499+0000 7f2587115640 1 Processor -- start 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.499+0000 7f2587115640 1 -- start start 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2580073310 0x7f25800cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f25801426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f258015e990 con 0x7f258015e5b0 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25800d16c0 con 0x7f2580073310 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25800b4030 con 0x7f25800d1be0 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51064/0 (socket says 172.21.15.181:51064) 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 -- 172.21.15.181:0/2780354049 learned_addr learned my addr 172.21.15.181:0/2780354049 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2584eb1640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2580073310 0x7f25800cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f257ffff640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f25801426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 -- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 msgr2=0x7f25801426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f25801426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 -- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2580073310 msgr2=0x7f25800cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2580073310 0x7f25800cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 -- 172.21.15.181:0/2780354049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2580160fa0 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f25856b2640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f257800efc0 tx=0x7f257800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578018020 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2578014480 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578014870 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2580161280 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.500+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2580161840 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.501+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2580075890 con 0x7f258015e5b0 2024-03-29T04:09:54.482 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.502+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2578005d10 con 0x7f258015e5b0 2024-03-29T04:09:54.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.502+0000 7f257d7fa640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 0x7f256406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.502+0000 7f2584eb1640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 0x7f256406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.502+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f257808c6c0 con 0x7f258015e5b0 2024-03-29T04:09:54.483 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.503+0000 7f2584eb1640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 0x7f256406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2574009560 tx=0x7f2574009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.507+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2578022030 con 0x7f258015e5b0 2024-03-29T04:09:54.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.681+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2580072be0 con 0x7f258015e5b0 2024-03-29T04:09:54.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.681+0000 7f257d7fa640 1 -- 172.21.15.181:0/2780354049 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f2578010040 con 0x7f258015e5b0 2024-03-29T04:09:54.660 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 msgr2=0x7f256406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 0x7f256406e9f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2574009560 tx=0x7f2574009190 comp rx=0 tx=0).stop 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 msgr2=0x7f2580142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f257800efc0 tx=0x7f257800c490 comp rx=0 tx=0).stop 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 shutdown_connections 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258015e5b0 0x7f2580142c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f256406c530 0x7f256406e9f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25800d1be0 0x7f25801426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 --2- 172.21.15.181:0/2780354049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2580073310 0x7f25800cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 >> 172.21.15.181:0/2780354049 conn(0x7f258006e720 msgr2=0x7f25800b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 shutdown_connections 2024-03-29T04:09:54.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.683+0000 7f2587115640 1 -- 172.21.15.181:0/2780354049 wait complete. 2024-03-29T04:09:54.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:54.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 Processor -- start 2024-03-29T04:09:54.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 -- start start 2024-03-29T04:09:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41e40b5dc0 0x7f41e40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41e40ba090 0x7f41e40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e40b66e0 con 0x7f41e406c620 2024-03-29T04:09:54.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f41e40d2ff0 con 0x7f41e40b5dc0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.798+0000 7f41ea5a8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f41e40d32d0 con 0x7f41e40ba090 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41e40b5dc0 0x7f41e40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f41e40b5dc0 0x7f41e40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51080/0 (socket says 172.21.15.181:51080) 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e8b45640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f41e406c620 0x7f41e406ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37010/0 (socket says 172.21.15.181:37010) 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 -- 172.21.15.181:0/1164521408 learned_addr learned my addr 172.21.15.181:0/1164521408 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41dbfff640 1 --2- 172.21.15.181:0/1164521408 >> v2:172.21.15.187:3300/0 conn(0x7f41e40ba090 0x7f41e40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 -- 172.21.15.181:0/1164521408 >> v2:172.21.15.187:3300/0 conn(0x7f41e40ba090 msgr2=0x7f41e40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 --2- 172.21.15.181:0/1164521408 >> v2:172.21.15.187:3300/0 conn(0x7f41e40ba090 0x7f41e40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 -- 172.21.15.181:0/1164521408 >> v1:172.21.15.181:6789/0 conn(0x7f41e406c620 legacy=0x7f41e406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e3fff640 1 -- 172.21.15.181:0/1164521408 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e40d35b0 con 0x7f41e40b5dc0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.799+0000 7f41e37fe640 1 -- 172.21.15.181:0/1164521408 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3027186019 0 0) 0x7f41e40b66e0 con 0x7f41e406c620 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41e3fff640 1 --2- 172.21.15.181:0/1164521408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e40b5dc0 0x7f41e40b61a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41e40b7240 tx=0x7f41d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c7a66535ded17e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41e37fe640 1 -- 172.21.15.181:0/1164521408 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4034070 con 0x7f41e40b5dc0 2024-03-29T04:09:54.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41e37fe640 1 -- 172.21.15.181:0/1164521408 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d4030860 con 0x7f41e40b5dc0 2024-03-29T04:09:54.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41e37fe640 1 -- 172.21.15.181:0/1164521408 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4030c30 con 0x7f41e40b5dc0 2024-03-29T04:09:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 -- 172.21.15.181:0/1164521408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e40b5dc0 msgr2=0x7f41e40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/1164521408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e40b5dc0 0x7f41e40b61a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41e40b7240 tx=0x7f41d402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 -- 172.21.15.181:0/1164521408 shutdown_connections 2024-03-29T04:09:54.783 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/1164521408 >> v2:172.21.15.187:3300/0 conn(0x7f41e40ba090 0x7f41e40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/1164521408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e40b5dc0 0x7f41e40b61a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 -- 172.21.15.181:0/1164521408 >> 172.21.15.181:0/1164521408 conn(0x7f41e40678f0 msgr2=0x7f41e40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 -- 172.21.15.181:0/1164521408 shutdown_connections 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.800+0000 7f41ea5a8640 1 -- 172.21.15.181:0/1164521408 wait complete. 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 Processor -- start 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 -- start start 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41e40b5dc0 0x7f41e40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e40ba090 0x7f41e415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d4090 con 0x7f41e406c620 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41e40d3b70 con 0x7f41e40b5dc0 2024-03-29T04:09:54.784 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41ea5a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41e40b83e0 con 0x7f41e40ba090 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41e3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41e3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51090/0 (socket says 172.21.15.181:51090) 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41e3fff640 1 -- 172.21.15.181:0/346221213 learned_addr learned my addr 172.21.15.181:0/346221213 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41dbfff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41e40b5dc0 0x7f41e40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.802+0000 7f41e8b45640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e40ba090 0x7f41e415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 -- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e40ba090 msgr2=0x7f41e415e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e40ba090 0x7f41e415e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 -- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41e40b5dc0 msgr2=0x7f41e40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41e40b5dc0 0x7f41e40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 -- 172.21.15.181:0/346221213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e415f0b0 con 0x7f41e406c620 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e3fff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f41d402b9e0 tx=0x7f41d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4048070 con 0x7f41e406c620 2024-03-29T04:09:54.785 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d4030660 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e415f390 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4034040 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.803+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e415f950 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.804+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e4075890 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.805+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41d404b070 con 0x7f41e406c620 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.805+0000 7f41e17fa640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 0x7f41c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.805+0000 7f41dbfff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 0x7f41c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:54.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.805+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f41d40ad820 con 0x7f41e406c620 2024-03-29T04:09:54.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.805+0000 7f41dbfff640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 0x7f41c406e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41e4161950 tx=0x7f41cc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:54.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.809+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41d407b540 con 0x7f41e406c620 2024-03-29T04:09:54.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.981+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41e41619c0 con 0x7f41e406c620 2024-03-29T04:09:54.961 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.981+0000 7f41e17fa640 1 -- 172.21.15.181:0/346221213 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f41d407f840 con 0x7f41e406c620 2024-03-29T04:09:54.961 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:54.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 msgr2=0x7f41c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 0x7f41c406e9f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41e4161950 tx=0x7f41cc009190 comp rx=0 tx=0).stop 2024-03-29T04:09:54.963 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 msgr2=0x7f41e415fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f41d402b9e0 tx=0x7f41d4004730 comp rx=0 tx=0).stop 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 shutdown_connections 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f41c406c530 0x7f41c406e9f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41e40ba090 0x7f41e415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41e40b5dc0 0x7f41e40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 --2- 172.21.15.181:0/346221213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41e406c620 0x7f41e415fc00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 >> 172.21.15.181:0/346221213 conn(0x7f41e40678f0 msgr2=0x7f41e40b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 shutdown_connections 2024-03-29T04:09:54.964 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:54.984+0000 7f41ea5a8640 1 -- 172.21.15.181:0/346221213 wait complete. 2024-03-29T04:09:54.974 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:55.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.098+0000 7f65aa04a640 1 Processor -- start 2024-03-29T04:09:55.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.098+0000 7f65aa04a640 1 -- start start 2024-03-29T04:09:55.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65aa04a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 0x7f65a40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65aa04a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65a40ba090 0x7f65a40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65aa04a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a40b66e0 con 0x7f65a406c620 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65aa04a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40d2ff0 con 0x7f65a40ba090 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65aa04a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40d32d0 con 0x7f65a40b5dc0 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65a40ba090 0x7f65a40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a37fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 0x7f65a40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65a40ba090 0x7f65a40d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51092/0 (socket says 172.21.15.181:51092) 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 -- 172.21.15.181:0/1006458577 learned_addr learned my addr 172.21.15.181:0/1006458577 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 -- 172.21.15.181:0/1006458577 >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 msgr2=0x7f65a40b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:55.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 --2- 172.21.15.181:0/1006458577 >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 0x7f65a40b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 -- 172.21.15.181:0/1006458577 >> v1:172.21.15.187:6789/0 conn(0x7f65a406c620 legacy=0x7f65a406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a2ffd640 1 -- 172.21.15.181:0/1006458577 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40d35b0 con 0x7f65a40ba090 2024-03-29T04:09:55.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.099+0000 7f65a37fe640 1 --2- 172.21.15.181:0/1006458577 >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 0x7f65a40b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:55.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.100+0000 7f65a2ffd640 1 --2- 172.21.15.181:0/1006458577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40ba090 0x7f65a40d2680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65a406d770 tx=0x7f659802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaeea562079fb0da server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.100+0000 7f65a27fc640 1 -- 172.21.15.181:0/1006458577 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598034070 con 0x7f65a40ba090 2024-03-29T04:09:55.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.100+0000 7f65a27fc640 1 -- 172.21.15.181:0/1006458577 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598030860 con 0x7f65a40ba090 2024-03-29T04:09:55.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.100+0000 7f65a27fc640 1 -- 172.21.15.181:0/1006458577 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598030c30 con 0x7f65a40ba090 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 -- 172.21.15.181:0/1006458577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40ba090 msgr2=0x7f65a40d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 --2- 172.21.15.181:0/1006458577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40ba090 0x7f65a40d2680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65a406d770 tx=0x7f659802b9a0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 -- 172.21.15.181:0/1006458577 shutdown_connections 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 --2- 172.21.15.181:0/1006458577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40ba090 0x7f65a40d2680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 --2- 172.21.15.181:0/1006458577 >> v2:172.21.15.187:3300/0 conn(0x7f65a40b5dc0 0x7f65a40b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 -- 172.21.15.181:0/1006458577 >> 172.21.15.181:0/1006458577 conn(0x7f65a40678f0 msgr2=0x7f65a40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 -- 172.21.15.181:0/1006458577 shutdown_connections 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.101+0000 7f65aa04a640 1 -- 172.21.15.181:0/1006458577 wait complete. 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.102+0000 7f65aa04a640 1 Processor -- start 2024-03-29T04:09:55.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.102+0000 7f65aa04a640 1 -- start start 2024-03-29T04:09:55.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a406c620 0x7f65a415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ba090 0x7f65a40ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d4090 con 0x7f65a40b5dc0 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d3b70 con 0x7f65a406c620 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65aa04a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40b83e0 con 0x7f65a40ba090 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51108/0 (socket says 172.21.15.181:51108) 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 -- 172.21.15.181:0/3375872323 learned_addr learned my addr 172.21.15.181:0/3375872323 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a37fe640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a406c620 0x7f65a415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a3fff640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ba090 0x7f65a40ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 -- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ba090 msgr2=0x7f65a40ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ba090 0x7f65a40ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 -- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a406c620 msgr2=0x7f65a415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a406c620 0x7f65a415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 -- 172.21.15.181:0/3375872323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a41f7da0 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.103+0000 7f65a2ffd640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f659802b9e0 tx=0x7f65980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.104+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598034040 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.104+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659803ad20 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.104+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a41f8080 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.104+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598044610 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.104+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a41f8640 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.105+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a4075890 con 0x7f65a40b5dc0 2024-03-29T04:09:55.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.105+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f659804b060 con 0x7f65a40b5dc0 2024-03-29T04:09:55.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.105+0000 7f658bfff640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 0x7f659006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.105+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f65980ad930 con 0x7f65a40b5dc0 2024-03-29T04:09:55.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.106+0000 7f65a37fe640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 0x7f659006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.106+0000 7f65a37fe640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 0x7f659006e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f65a40cdf40 tx=0x7f6594009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.110+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f659803f070 con 0x7f65a40b5dc0 2024-03-29T04:09:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.285+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65a40cf040 con 0x7f65a40b5dc0 2024-03-29T04:09:55.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.286+0000 7f658bfff640 1 -- 172.21.15.181:0/3375872323 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f659807b650 con 0x7f65a40b5dc0 2024-03-29T04:09:55.265 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 msgr2=0x7f659006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 0x7f659006e9f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f65a40cdf40 tx=0x7f6594009190 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 msgr2=0x7f65a4157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f659802b9e0 tx=0x7f65980046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 shutdown_connections 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f659006c530 0x7f659006e9f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65a40ba090 0x7f65a40ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a40b5dc0 0x7f65a4157610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 --2- 172.21.15.181:0/3375872323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a406c620 0x7f65a415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.268 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 >> 172.21.15.181:0/3375872323 conn(0x7f65a40678f0 msgr2=0x7f65a40b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 shutdown_connections 2024-03-29T04:09:55.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.288+0000 7f65aa04a640 1 -- 172.21.15.181:0/3375872323 wait complete. 2024-03-29T04:09:55.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:55.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.403+0000 7fbc98ab7640 1 Processor -- start 2024-03-29T04:09:55.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.403+0000 7fbc98ab7640 1 -- start start 2024-03-29T04:09:55.384 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc98ab7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc98ab7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc940d0270 0x7fbc9409af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc98ab7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc940cd270 con 0x7fbc940b4a10 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc98ab7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbc940cd550 con 0x7fbc940d0270 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc98ab7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc940cd830 con 0x7fbc9415c500 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc91d9b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbc940b4a10 0x7fbc940b0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37024/0 (socket says 172.21.15.181:37024) 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc91d9b640 1 -- 172.21.15.181:0/2170073819 learned_addr learned my addr 172.21.15.181:0/2170073819 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9259c640 1 --2- 172.21.15.181:0/2170073819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc92d9d640 1 --2- 172.21.15.181:0/2170073819 >> v2:172.21.15.187:3300/0 conn(0x7fbc940d0270 0x7fbc9409af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9159a640 1 -- 172.21.15.181:0/2170073819 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482829712 0 0) 0x7fbc940cd270 con 0x7fbc940b4a10 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9159a640 1 -- 172.21.15.181:0/2170073819 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc7c003520 con 0x7fbc940b4a10 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9259c640 1 -- 172.21.15.181:0/2170073819 >> v2:172.21.15.187:3300/0 conn(0x7fbc940d0270 msgr2=0x7fbc9409af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:55.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9259c640 1 --2- 172.21.15.181:0/2170073819 >> v2:172.21.15.187:3300/0 conn(0x7fbc940d0270 0x7fbc9409af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9259c640 1 -- 172.21.15.181:0/2170073819 >> v1:172.21.15.181:6789/0 conn(0x7fbc940b4a10 legacy=0x7fbc940b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.404+0000 7fbc9259c640 1 -- 172.21.15.181:0/2170073819 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc940cdb10 con 0x7fbc9415c500 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.405+0000 7fbc9259c640 1 --2- 172.21.15.181:0/2170073819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9409aa30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097a0 tx=0x7fbc8402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85df2d6766c129a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.405+0000 7fbc9159a640 1 -- 172.21.15.181:0/2170073819 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84034070 con 0x7fbc9415c500 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.405+0000 7fbc9159a640 1 -- 172.21.15.181:0/2170073819 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc84030610 con 0x7fbc9415c500 2024-03-29T04:09:55.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.405+0000 7fbc9159a640 1 -- 172.21.15.181:0/2170073819 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc840309e0 con 0x7fbc9415c500 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 -- 172.21.15.181:0/2170073819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 msgr2=0x7fbc9409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/2170073819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9409aa30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097a0 tx=0x7fbc8402b960 comp rx=0 tx=0).stop 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 -- 172.21.15.181:0/2170073819 shutdown_connections 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/2170073819 >> v2:172.21.15.187:3300/0 conn(0x7fbc940d0270 0x7fbc9409af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/2170073819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9409aa30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 -- 172.21.15.181:0/2170073819 >> 172.21.15.181:0/2170073819 conn(0x7fbc940b3ce0 msgr2=0x7fbc940b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 -- 172.21.15.181:0/2170073819 shutdown_connections 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.406+0000 7fbc98ab7640 1 -- 172.21.15.181:0/2170073819 wait complete. 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.407+0000 7fbc98ab7640 1 Processor -- start 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.407+0000 7fbc98ab7640 1 -- start start 2024-03-29T04:09:55.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 0x7fbc94153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940b1590 con 0x7fbc940d0270 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940ccee0 con 0x7fbc940b4a10 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc98ab7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc940b1710 con 0x7fbc9415c500 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51118/0 (socket says 172.21.15.181:51118) 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 -- 172.21.15.181:0/1998886432 learned_addr learned my addr 172.21.15.181:0/1998886432 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc9259c640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 0x7fbc94153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc92d9d640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 -- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 msgr2=0x7fbc9415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9415c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 -- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 msgr2=0x7fbc94153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 0x7fbc94153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 -- 172.21.15.181:0/1998886432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc940b0670 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc9259c640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 0x7fbc94153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc91d9b640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c430 tx=0x7fbc8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.408+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88017070 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.409+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc88005ce0 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.409+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc940b0950 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.409+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880072f0 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.409+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc940b6210 con 0x7fbc940d0270 2024-03-29T04:09:55.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.409+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc940b6410 con 0x7fbc940d0270 2024-03-29T04:09:55.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.410+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbc8801a070 con 0x7fbc940d0270 2024-03-29T04:09:55.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.410+0000 7fbc82ffd640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 0x7fbc7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.410+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbc8808ad70 con 0x7fbc940d0270 2024-03-29T04:09:55.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.410+0000 7fbc9259c640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 0x7fbc7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.411+0000 7fbc9259c640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 0x7fbc7406e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc84036000 tx=0x7fbc84008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.415+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbc940b6410 con 0x7fbc940d0270 2024-03-29T04:09:55.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.590+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbc94075890 con 0x7fbc940d0270 2024-03-29T04:09:55.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.591+0000 7fbc82ffd640 1 -- 172.21.15.181:0/1998886432 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fbc88058a90 con 0x7fbc940d0270 2024-03-29T04:09:55.570 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:55.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 msgr2=0x7fbc7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 0x7fbc7406e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc84036000 tx=0x7fbc84008030 comp rx=0 tx=0).stop 2024-03-29T04:09:55.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 msgr2=0x7fbc940affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c430 tx=0x7fbc8800c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.572 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 shutdown_connections 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbc7406c530 0x7fbc7406e9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc9415c500 0x7fbc9415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc940d0270 0x7fbc940affe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 --2- 172.21.15.181:0/1998886432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc940b4a10 0x7fbc94153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 >> 172.21.15.181:0/1998886432 conn(0x7fbc940b3ce0 msgr2=0x7fbc9415b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 shutdown_connections 2024-03-29T04:09:55.573 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.593+0000 7fbc98ab7640 1 -- 172.21.15.181:0/1998886432 wait complete. 2024-03-29T04:09:55.583 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:55.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.704+0000 7f77bde22640 1 Processor -- start 2024-03-29T04:09:55.684 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.704+0000 7f77bde22640 1 -- start start 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77bde22640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77b8072230 0x7f77b806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77bde22640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b806c4d0 con 0x7f77b806db10 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77bde22640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80ce840 con 0x7f77b8072610 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77bde22640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f77b8158f10 con 0x7f77b8072230 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77b8072230 0x7f77b806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b7fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f77b806db10 0x7f77b806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37040/0 (socket says 172.21.15.181:37040) 2024-03-29T04:09:55.685 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77b8072230 0x7f77b806bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51120/0 (socket says 172.21.15.181:51120) 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b7fff640 1 -- 172.21.15.181:0/2552587252 learned_addr learned my addr 172.21.15.181:0/2552587252 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b6ffd640 1 --1- 172.21.15.181:0/2552587252 >> v1:172.21.15.187:6789/0 conn(0x7f77b8072610 0x7f77b815f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59442/0 (socket says 172.21.15.181:59442) 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b67fc640 1 -- 172.21.15.181:0/2552587252 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262786761 0 0) 0x7f77b80ce840 con 0x7f77b8072610 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b77fe640 1 -- 172.21.15.181:0/2552587252 >> v1:172.21.15.187:6789/0 conn(0x7f77b8072610 legacy=0x7f77b815f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b77fe640 1 -- 172.21.15.181:0/2552587252 >> v1:172.21.15.181:6789/0 conn(0x7f77b806db10 legacy=0x7f77b806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.705+0000 7f77b77fe640 1 -- 172.21.15.181:0/2552587252 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b80ceb20 con 0x7f77b8072230 2024-03-29T04:09:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.706+0000 7f77b77fe640 1 --2- 172.21.15.181:0/2552587252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b806bf90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f77a80097a0 tx=0x7f77a802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdffef1f63742f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.706+0000 7f77b67fc640 1 -- 172.21.15.181:0/2552587252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8037070 con 0x7f77b8072230 2024-03-29T04:09:55.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.706+0000 7f77b67fc640 1 -- 172.21.15.181:0/2552587252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a8030710 con 0x7f77b8072230 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.706+0000 7f77b67fc640 1 -- 172.21.15.181:0/2552587252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8030ae0 con 0x7f77b8072230 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 -- 172.21.15.181:0/2552587252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 msgr2=0x7f77b806bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 --2- 172.21.15.181:0/2552587252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b806bf90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f77a80097a0 tx=0x7f77a802b790 comp rx=0 tx=0).stop 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 -- 172.21.15.181:0/2552587252 shutdown_connections 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 --2- 172.21.15.181:0/2552587252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b806bf90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 -- 172.21.15.181:0/2552587252 >> 172.21.15.181:0/2552587252 conn(0x7f77b80d0cb0 msgr2=0x7f77b80d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 -- 172.21.15.181:0/2552587252 shutdown_connections 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.707+0000 7f77bde22640 1 -- 172.21.15.181:0/2552587252 wait complete. 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.708+0000 7f77bde22640 1 Processor -- start 2024-03-29T04:09:55.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.708+0000 7f77bde22640 1 -- start start 2024-03-29T04:09:55.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77b806db10 0x7f77b814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77b8072610 0x7f77b81558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f77b806e630 con 0x7f77b8072230 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77b815ef70 con 0x7f77b806db10 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77bde22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f77b8072ac0 con 0x7f77b8072610 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51134/0 (socket says 172.21.15.181:51134) 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 -- 172.21.15.181:0/3006305234 learned_addr learned my addr 172.21.15.181:0/3006305234 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b77fe640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77b806db10 0x7f77b814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b7fff640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77b8072610 0x7f77b81558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 -- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77b8072610 msgr2=0x7f77b81558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77b8072610 0x7f77b81558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 -- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77b806db10 msgr2=0x7f77b814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77b806db10 0x7f77b814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 -- 172.21.15.181:0/3006305234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b814d8c0 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.709+0000 7f77b6ffd640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c430 tx=0x7f77ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.710+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0074e0 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.710+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac0047e0 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.710+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b8160370 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.710+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac004bd0 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.710+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b8160c10 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.711+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b8075a10 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.711+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f77ac011070 con 0x7f77b8072230 2024-03-29T04:09:55.692 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.711+0000 7f779ffff640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 0x7f77a406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.712+0000 7f77b77fe640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 0x7f77a406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.712+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f77ac08b470 con 0x7f77b8072230 2024-03-29T04:09:55.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.712+0000 7f77b77fe640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 0x7f77a406e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77a8034000 tx=0x7f77a802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.695 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.716+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f77ac059190 con 0x7f77b8072230 2024-03-29T04:09:55.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.892+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77b814db10 con 0x7f77b8072230 2024-03-29T04:09:55.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.893+0000 7f779ffff640 1 -- 172.21.15.181:0/3006305234 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18526 (secure 0 0 0) 0x7f77ac05d490 con 0x7f77b8072230 2024-03-29T04:09:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:55.873 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":25,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:22.325995+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 msgr2=0x7f77a406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 0x7f77a406e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f77a8034000 tx=0x7f77a802f000 comp rx=0 tx=0).stop 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 msgr2=0x7f77b80b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c430 tx=0x7f77ac00c8f0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 shutdown_connections 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f77a406c530 0x7f77a406e9f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f77b8072610 0x7f77b81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77b8072230 0x7f77b80b5fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 --2- 172.21.15.181:0/3006305234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77b806db10 0x7f77b814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 >> 172.21.15.181:0/3006305234 conn(0x7f77b80d0cb0 msgr2=0x7f77b8074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 shutdown_connections 2024-03-29T04:09:55.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:55.895+0000 7f77bde22640 1 -- 172.21.15.181:0/3006305234 wait complete. 2024-03-29T04:09:55.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:55.887 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.011+0000 7fb37eb6d640 1 Processor -- start 2024-03-29T04:09:55.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.011+0000 7fb37eb6d640 1 -- start start 2024-03-29T04:09:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37eb6d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3780ba090 0x7fb3780d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37eb6d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3780d2ff0 con 0x7fb3780b73d0 2024-03-29T04:09:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37eb6d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3780d32d0 con 0x7fb3780b5dc0 2024-03-29T04:09:55.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37eb6d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780d35b0 con 0x7fb3780ba090 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb377fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3780ba090 0x7fb3780d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37d10a640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb3780b73d0 0x7fb3780b7820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37056/0 (socket says 172.21.15.181:37056) 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37d10a640 1 -- 172.21.15.181:0/428404377 learned_addr learned my addr 172.21.15.181:0/428404377 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb377fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3780ba090 0x7fb3780d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51148/0 (socket says 172.21.15.181:51148) 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.012+0000 7fb37c909640 1 --1- 172.21.15.181:0/428404377 >> v1:172.21.15.187:6789/0 conn(0x7fb3780b5dc0 0x7fb3780b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59458/0 (socket says 172.21.15.181:59458) 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb3777fe640 1 -- 172.21.15.181:0/428404377 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 409038802 0 0) 0x7fb3780d32d0 con 0x7fb3780b5dc0 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb377fff640 1 -- 172.21.15.181:0/428404377 >> v1:172.21.15.187:6789/0 conn(0x7fb3780b5dc0 legacy=0x7fb3780b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb377fff640 1 -- 172.21.15.181:0/428404377 >> v1:172.21.15.181:6789/0 conn(0x7fb3780b73d0 legacy=0x7fb3780b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb377fff640 1 -- 172.21.15.181:0/428404377 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780d3890 con 0x7fb3780ba090 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb377fff640 1 --2- 172.21.15.181:0/428404377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780ba090 0x7fb3780d2680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097d0 tx=0x7fb36c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5aacc5e4a19cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb3777fe640 1 -- 172.21.15.181:0/428404377 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c037070 con 0x7fb3780ba090 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb3777fe640 1 -- 172.21.15.181:0/428404377 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c030710 con 0x7fb3780ba090 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.013+0000 7fb3777fe640 1 -- 172.21.15.181:0/428404377 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c030ae0 con 0x7fb3780ba090 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 -- 172.21.15.181:0/428404377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780ba090 msgr2=0x7fb3780d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/428404377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780ba090 0x7fb3780d2680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0097d0 tx=0x7fb36c02b790 comp rx=0 tx=0).stop 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 -- 172.21.15.181:0/428404377 shutdown_connections 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/428404377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780ba090 0x7fb3780d2680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 -- 172.21.15.181:0/428404377 >> 172.21.15.181:0/428404377 conn(0x7fb3780678f0 msgr2=0x7fb3780d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 -- 172.21.15.181:0/428404377 shutdown_connections 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.014+0000 7fb37eb6d640 1 -- 172.21.15.181:0/428404377 wait complete. 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.015+0000 7fb37eb6d640 1 Processor -- start 2024-03-29T04:09:55.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.015+0000 7fb37eb6d640 1 -- start start 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3780b5dc0 0x7fb378148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3780ba090 0x7fb378159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb37815e880 con 0x7fb3780b73d0 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780d4090 con 0x7fb3780ba090 2024-03-29T04:09:55.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37eb6d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb37815ea00 con 0x7fb3780b5dc0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51158/0 (socket says 172.21.15.181:51158) 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 -- 172.21.15.181:0/3199704034 learned_addr learned my addr 172.21.15.181:0/3199704034 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37d10a640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3780ba090 0x7fb378159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb37c909640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3780b5dc0 0x7fb378148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 -- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3780b5dc0 msgr2=0x7fb378148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3780b5dc0 0x7fb378148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 -- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3780ba090 msgr2=0x7fb378159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3780ba090 0x7fb378159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.016+0000 7fb377fff640 1 -- 172.21.15.181:0/3199704034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378149480 con 0x7fb3780b73d0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb377fff640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0098f0 tx=0x7fb36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c042070 con 0x7fb3780b73d0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c032a40 con 0x7fb3780b73d0 2024-03-29T04:09:55.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37815a3e0 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c037040 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.017+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb378160590 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.018+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378075890 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.018+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb36c045070 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.018+0000 7fb3757fa640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 0x7fb36406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.019+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb36c0ad530 con 0x7fb3780b73d0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.019+0000 7fb37c909640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 0x7fb36406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.019+0000 7fb37c909640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 0x7fb36406e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb368000f60 tx=0x7fb368007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.023+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb36c07b2e0 con 0x7fb3780b73d0 2024-03-29T04:09:56.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.200+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3780caca0 con 0x7fb3780b73d0 2024-03-29T04:09:56.180 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.200+0000 7fb3757fa640 1 -- 172.21.15.181:0/3199704034 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fb36c07f5e0 con 0x7fb3780b73d0 2024-03-29T04:09:56.180 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:56.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.202+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 msgr2=0x7fb36406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.202+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 0x7fb36406e9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb368000f60 tx=0x7fb368007d50 comp rx=0 tx=0).stop 2024-03-29T04:09:56.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.202+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 msgr2=0x7fb3780c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.202+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0098f0 tx=0x7fb36c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 shutdown_connections 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3780ba090 0x7fb378159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3780b73d0 0x7fb3780c9e50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb36406c530 0x7fb36406e9f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 --2- 172.21.15.181:0/3199704034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3780b5dc0 0x7fb378148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 >> 172.21.15.181:0/3199704034 conn(0x7fb3780678f0 msgr2=0x7fb3780ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 shutdown_connections 2024-03-29T04:09:56.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.203+0000 7fb37eb6d640 1 -- 172.21.15.181:0/3199704034 wait complete. 2024-03-29T04:09:56.193 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.318+0000 7fc1e66f2640 1 Processor -- start 2024-03-29T04:09:56.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.318+0000 7fc1e66f2640 1 -- start start 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e66f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e015d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e66f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 0x7fc1e015fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e66f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e015c990 con 0x7fc1e0071aa0 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e66f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e0071f20 con 0x7fc1e00d1980 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e66f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00ccb70 con 0x7fc1e00b7950 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1df7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc1e0071aa0 0x7fc1e00d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:37072/0 (socket says 172.21.15.181:37072) 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1df7fe640 1 -- 172.21.15.181:0/2638350401 learned_addr learned my addr 172.21.15.181:0/2638350401 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1e4c8f640 1 --2- 172.21.15.181:0/2638350401 >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 0x7fc1e015fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2638350401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1deffd640 1 -- 172.21.15.181:0/2638350401 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293546200 0 0) 0x7fc1e015c990 con 0x7fc1e0071aa0 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.319+0000 7fc1deffd640 1 -- 172.21.15.181:0/2638350401 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c8003520 con 0x7fc1e0071aa0 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1dffff640 1 -- 172.21.15.181:0/2638350401 >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 msgr2=0x7fc1e015fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2638350401 >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 0x7fc1e015fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1dffff640 1 -- 172.21.15.181:0/2638350401 >> v1:172.21.15.181:6789/0 conn(0x7fc1e0071aa0 legacy=0x7fc1e00d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1dffff640 1 -- 172.21.15.181:0/2638350401 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e015cc00 con 0x7fc1e00b7950 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1e4c8f640 1 --2- 172.21.15.181:0/2638350401 >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 0x7fc1e015fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2638350401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e015d400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919a20d2f55bd6e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1deffd640 1 -- 172.21.15.181:0/2638350401 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0035020 con 0x7fc1e00b7950 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1deffd640 1 -- 172.21.15.181:0/2638350401 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d0031830 con 0x7fc1e00b7950 2024-03-29T04:09:56.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.320+0000 7fc1deffd640 1 -- 172.21.15.181:0/2638350401 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0031c00 con 0x7fc1e00b7950 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2638350401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 msgr2=0x7fc1e015d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2638350401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e015d400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2638350401 shutdown_connections 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2638350401 >> v2:172.21.15.187:3300/0 conn(0x7fc1e00d1980 0x7fc1e015fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2638350401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e015d400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2638350401 >> 172.21.15.181:0/2638350401 conn(0x7fc1e00b3cb0 msgr2=0x7fc1e00b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2638350401 shutdown_connections 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.321+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2638350401 wait complete. 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.322+0000 7fc1e66f2640 1 Processor -- start 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.322+0000 7fc1e66f2640 1 -- start start 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e0071aa0 0x7fc1e01576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e00cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e0070790 con 0x7fc1e00d1980 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e00b7d30 con 0x7fc1e0071aa0 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e66f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00cc7b0 con 0x7fc1e00b7950 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51166/0 (socket says 172.21.15.181:51166) 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 -- 172.21.15.181:0/2791276644 learned_addr learned my addr 172.21.15.181:0/2791276644 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e0071aa0 0x7fc1e01576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1df7fe640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e00cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 -- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 msgr2=0x7fc1e00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e00cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 -- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e0071aa0 msgr2=0x7fc1e01576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e0071aa0 0x7fc1e01576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.323+0000 7fc1e4c8f640 1 -- 172.21.15.181:0/2791276644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e0154ef0 con 0x7fc1e00d1980 2024-03-29T04:09:56.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1e4c8f640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800efb0 tx=0x7fc1d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8018020 con 0x7fc1e00d1980 2024-03-29T04:09:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d8014480 con 0x7fc1e00d1980 2024-03-29T04:09:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e0157c00 con 0x7fc1e00d1980 2024-03-29T04:09:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8014850 con 0x7fc1e00d1980 2024-03-29T04:09:56.306 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.324+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e00b56f0 con 0x7fc1e00d1980 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.325+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0075890 con 0x7fc1e00d1980 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.325+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc1d8005d10 con 0x7fc1e00d1980 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.325+0000 7fc1dcff9640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.326+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1d808c690 con 0x7fc1e00d1980 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.326+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.307 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.326+0000 7fc1dffff640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.309 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.330+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc1d8022030 con 0x7fc1e00d1980 2024-03-29T04:09:56.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.512+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1e0156920 con 0x7fc1e00d1980 2024-03-29T04:09:56.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.512+0000 7fc1dcff9640 1 -- 172.21.15.181:0/2791276644 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc1d8010040 con 0x7fc1e00d1980 2024-03-29T04:09:56.493 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 msgr2=0x7fc1cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).stop 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 msgr2=0x7fc1e00c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800efb0 tx=0x7fc1d800c490 comp rx=0 tx=0).stop 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 shutdown_connections 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e00d1980 0x7fc1e00c5c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b7950 0x7fc1e00cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 --2- 172.21.15.181:0/2791276644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e0071aa0 0x7fc1e01576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 >> 172.21.15.181:0/2791276644 conn(0x7fc1e00b3cb0 msgr2=0x7fc1e00b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 shutdown_connections 2024-03-29T04:09:56.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.514+0000 7fc1e66f2640 1 -- 172.21.15.181:0/2791276644 wait complete. 2024-03-29T04:09:56.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:56.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.629+0000 7f289b0c8640 1 Processor -- start 2024-03-29T04:09:56.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.629+0000 7f289b0c8640 1 -- start start 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f289b0c8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f289b0c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 0x7f28940b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f289b0c8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2894162610 con 0x7f28940b3c90 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f289b0c8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f289406fac0 con 0x7f28940afd60 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f289b0c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28940b40d0 con 0x7f2894162070 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2898e64640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2893fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 0x7f28940b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2898e64640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:59954/0 (socket says 172.21.15.181:59954) 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2899665640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f28940b3c90 0x7f289406e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59466/0 (socket says 172.21.15.181:59466) 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2898e64640 1 -- 172.21.15.181:0/1970540823 learned_addr learned my addr 172.21.15.181:0/1970540823 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2893fff640 1 -- 172.21.15.181:0/1970540823 >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 msgr2=0x7f289415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2893fff640 1 --2- 172.21.15.181:0/1970540823 >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2893fff640 1 -- 172.21.15.181:0/1970540823 >> v1:172.21.15.187:6789/0 conn(0x7f28940b3c90 legacy=0x7f289406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2893fff640 1 -- 172.21.15.181:0/1970540823 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28941628f0 con 0x7f2894162070 2024-03-29T04:09:56.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.630+0000 7f2898e64640 1 --2- 172.21.15.181:0/1970540823 >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:56.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.631+0000 7f2893fff640 1 --2- 172.21.15.181:0/1970540823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 0x7f28940b3750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28800097a0 tx=0x7f288002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2bc8422222fcc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.631+0000 7f28937fe640 1 -- 172.21.15.181:0/1970540823 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880035020 con 0x7f2894162070 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.631+0000 7f28937fe640 1 -- 172.21.15.181:0/1970540823 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2880031830 con 0x7f2894162070 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.631+0000 7f28937fe640 1 -- 172.21.15.181:0/1970540823 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880031c00 con 0x7f2894162070 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 -- 172.21.15.181:0/1970540823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 msgr2=0x7f28940b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1970540823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 0x7f28940b3750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28800097a0 tx=0x7f288002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 -- 172.21.15.181:0/1970540823 shutdown_connections 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1970540823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2894162070 0x7f28940b3750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1970540823 >> v2:172.21.15.187:3300/0 conn(0x7f28940afd60 0x7f289415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 -- 172.21.15.181:0/1970540823 >> 172.21.15.181:0/1970540823 conn(0x7f28940d0840 msgr2=0x7f28940d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 -- 172.21.15.181:0/1970540823 shutdown_connections 2024-03-29T04:09:56.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.632+0000 7f289b0c8640 1 -- 172.21.15.181:0/1970540823 wait complete. 2024-03-29T04:09:56.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.633+0000 7f289b0c8640 1 Processor -- start 2024-03-29T04:09:56.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.633+0000 7f289b0c8640 1 -- start start 2024-03-29T04:09:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28940afd60 0x7f28941586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2894162070 0x7f2894160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2894162eb0 con 0x7f28940b3c90 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28940b0140 con 0x7f2894162070 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f289406cb10 con 0x7f28940afd60 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51168/0 (socket says 172.21.15.181:51168) 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 -- 172.21.15.181:0/1992325828 learned_addr learned my addr 172.21.15.181:0/1992325828 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2898e64640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28940afd60 0x7f28941586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2899665640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2894162070 0x7f2894160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 -- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28940afd60 msgr2=0x7f28941586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28940afd60 0x7f28941586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 -- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2894162070 msgr2=0x7f2894160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2894162070 0x7f2894160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 -- 172.21.15.181:0/1992325828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2894158c30 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f2893fff640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2880034040 tx=0x7f28800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880040070 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288003a910 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880035040 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.634+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941f4550 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.635+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941f4b10 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.635+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2894075890 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.636+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2880038020 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.636+0000 7f28917fa640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 0x7f287806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.636+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f28800ad2f0 con 0x7f28940b3c90 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.636+0000 7f2898e64640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 0x7f287806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.637+0000 7f2898e64640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 0x7f287806e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2888006ca0 tx=0x7f2888009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.641+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2894158c30 con 0x7f28940b3c90 2024-03-29T04:09:56.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.816+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28940675e0 con 0x7f28940b3c90 2024-03-29T04:09:56.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.817+0000 7f28917fa640 1 -- 172.21.15.181:0/1992325828 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f2880031800 con 0x7f28940b3c90 2024-03-29T04:09:56.796 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:56.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 msgr2=0x7f287806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 0x7f287806e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2888006ca0 tx=0x7f2888009240 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 msgr2=0x7f2894143370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2880034040 tx=0x7f28800046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 shutdown_connections 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2894162070 0x7f2894160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28940b3c90 0x7f2894143370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f287806c530 0x7f287806e9f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 --2- 172.21.15.181:0/1992325828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28940afd60 0x7f28941586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 >> 172.21.15.181:0/1992325828 conn(0x7f28940d0840 msgr2=0x7f28940adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 shutdown_connections 2024-03-29T04:09:56.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.819+0000 7f289b0c8640 1 -- 172.21.15.181:0/1992325828 wait complete. 2024-03-29T04:09:56.809 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:56.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.933+0000 7f60647f4640 1 Processor -- start 2024-03-29T04:09:56.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.933+0000 7f60647f4640 1 -- start start 2024-03-29T04:09:56.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f60647f4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605c0b3e60 0x7f605c15f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f60647f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c162070 0x7f605c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f60647f4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605c162610 con 0x7f605c0afd60 2024-03-29T04:09:56.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f60647f4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f605c06fac0 con 0x7f605c0b3e60 2024-03-29T04:09:56.915 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f60647f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f605c1628f0 con 0x7f605c162070 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605c0b3e60 0x7f605c15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f605c0b3e60 0x7f605c15f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51170/0 (socket says 172.21.15.181:51170) 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 -- 172.21.15.181:0/1118102434 learned_addr learned my addr 172.21.15.181:0/1118102434 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6062590640 1 --1- 172.21.15.181:0/1118102434 >> v1:172.21.15.187:6789/0 conn(0x7f605c0afd60 0x7f605c0b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:59482/0 (socket says 172.21.15.181:59482) 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6062d91640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c162070 0x7f605c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 -- 172.21.15.181:0/1118102434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c162070 msgr2=0x7f605c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c162070 0x7f605c06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.916 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 -- 172.21.15.181:0/1118102434 >> v1:172.21.15.187:6789/0 conn(0x7f605c0afd60 legacy=0x7f605c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.934+0000 7f6061d8f640 1 -- 172.21.15.181:0/1118102434 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c162bd0 con 0x7f605c0b3e60 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.935+0000 7f6061d8f640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c0b3e60 0x7f605c15f500 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f604c009870 tx=0x7f604c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc764c20032a3659 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.935+0000 7f606158e640 1 -- 172.21.15.181:0/1118102434 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c034070 con 0x7f605c0b3e60 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.935+0000 7f606158e640 1 -- 172.21.15.181:0/1118102434 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c030840 con 0x7f605c0b3e60 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.935+0000 7f606158e640 1 -- 172.21.15.181:0/1118102434 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c030c10 con 0x7f605c0b3e60 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 -- 172.21.15.181:0/1118102434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c0b3e60 msgr2=0x7f605c15f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c0b3e60 0x7f605c15f500 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f604c009870 tx=0x7f604c02b980 comp rx=0 tx=0).stop 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 -- 172.21.15.181:0/1118102434 shutdown_connections 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c162070 0x7f605c06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 --2- 172.21.15.181:0/1118102434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c0b3e60 0x7f605c15f500 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 -- 172.21.15.181:0/1118102434 >> 172.21.15.181:0/1118102434 conn(0x7f605c0d0840 msgr2=0x7f605c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:56.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 -- 172.21.15.181:0/1118102434 shutdown_connections 2024-03-29T04:09:56.918 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.936+0000 7f60647f4640 1 -- 172.21.15.181:0/1118102434 wait complete. 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.937+0000 7f60647f4640 1 Processor -- start 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.937+0000 7f60647f4640 1 -- start start 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.937+0000 7f60647f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f605c0afd60 0x7f605c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.937+0000 7f60647f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c0b3e60 0x7f605c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f605c06c040 con 0x7f605c162070 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f605c162eb0 con 0x7f605c0afd60 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f605c06c1c0 con 0x7f605c0b3e60 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51172/0 (socket says 172.21.15.181:51172) 2024-03-29T04:09:56.921 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 -- 172.21.15.181:0/2815665377 learned_addr learned my addr 172.21.15.181:0/2815665377 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062590640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f605c0afd60 0x7f605c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6061d8f640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c0b3e60 0x7f605c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 -- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c0b3e60 msgr2=0x7f605c152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c0b3e60 0x7f605c152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 -- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f605c0afd60 msgr2=0x7f605c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f605c0afd60 0x7f605c1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 -- 172.21.15.181:0/2815665377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c1f7a50 con 0x7f605c162070 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6062d91640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f605400efc0 tx=0x7f605400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054018020 con 0x7f605c162070 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6054014480 con 0x7f605c162070 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c1f7d30 con 0x7f605c162070 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054014870 con 0x7f605c162070 2024-03-29T04:09:56.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.938+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c1f82f0 con 0x7f605c162070 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.939+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c075890 con 0x7f605c162070 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.940+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f605401b020 con 0x7f605c162070 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.940+0000 7f6052ffd640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 0x7f604006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.940+0000 7f6062590640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 0x7f604006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.940+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f605408c4c0 con 0x7f605c162070 2024-03-29T04:09:56.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.941+0000 7f6062590640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 0x7f604006e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6058000f60 tx=0x7f6058007e20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:56.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:56.945+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6054022030 con 0x7f605c162070 2024-03-29T04:09:57.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.119+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f605c0adbe0 con 0x7f605c162070 2024-03-29T04:09:57.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.120+0000 7f6052ffd640 1 -- 172.21.15.181:0/2815665377 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f6054010040 con 0x7f605c162070 2024-03-29T04:09:57.100 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:57.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 msgr2=0x7f604006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 0x7f604006e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6058000f60 tx=0x7f6058007e20 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 msgr2=0x7f605c1547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f605400efc0 tx=0x7f605400c490 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 shutdown_connections 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f605c162070 0x7f605c1547a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f604006c530 0x7f604006e9f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f605c0b3e60 0x7f605c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 --2- 172.21.15.181:0/2815665377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f605c0afd60 0x7f605c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 >> 172.21.15.181:0/2815665377 conn(0x7f605c0d0840 msgr2=0x7f605c0ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 shutdown_connections 2024-03-29T04:09:57.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.122+0000 7f60647f4640 1 -- 172.21.15.181:0/2815665377 wait complete. 2024-03-29T04:09:57.118 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:09:57.119 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:09:57.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.240+0000 7f3ce4f82640 1 Processor -- start 2024-03-29T04:09:57.220 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.240+0000 7f3ce4f82640 1 -- start start 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3ce4f82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3ce4f82640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ce0072230 0x7f3ce00726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3ce4f82640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ce015c600 con 0x7f3ce00d0360 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3ce4f82640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f3ce0072be0 con 0x7f3ce0072230 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3ce4f82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce006bdd0 con 0x7f3ce015c220 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cded9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ce0072230 0x7f3ce00726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cded9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ce0072230 0x7f3ce00726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51178/0 (socket says 172.21.15.181:51178) 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cded9d640 1 -- 172.21.15.181:0/2409562033 learned_addr learned my addr 172.21.15.181:0/2409562033 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:57.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cde59c640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cded9d640 1 -- 172.21.15.181:0/2409562033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 msgr2=0x7f3ce006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.241+0000 7f3cdd59a640 1 -- 172.21.15.181:0/2409562033 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872375899 0 0) 0x7f3ce015c600 con 0x7f3ce00d0360 2024-03-29T04:09:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cded9d640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cded9d640 1 -- 172.21.15.181:0/2409562033 >> v1:172.21.15.187:6789/0 conn(0x7f3ce00d0360 legacy=0x7f3ce00ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cded9d640 1 -- 172.21.15.181:0/2409562033 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce006c0b0 con 0x7f3ce0072230 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cded9d640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce00726a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009870 tx=0x7f3cd802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c43429c56bfff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cdd59a640 1 -- 172.21.15.181:0/2409562033 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8034070 con 0x7f3ce0072230 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cdd59a640 1 -- 172.21.15.181:0/2409562033 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd8030840 con 0x7f3ce0072230 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.242+0000 7f3cdd59a640 1 -- 172.21.15.181:0/2409562033 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8030c10 con 0x7f3ce0072230 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 -- 172.21.15.181:0/2409562033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 msgr2=0x7f3ce00726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce00726a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009870 tx=0x7f3cd802b980 comp rx=0 tx=0).stop 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 -- 172.21.15.181:0/2409562033 shutdown_connections 2024-03-29T04:09:57.223 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce00726a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/2409562033 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 -- 172.21.15.181:0/2409562033 >> 172.21.15.181:0/2409562033 conn(0x7f3ce00d0cb0 msgr2=0x7f3ce00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 -- 172.21.15.181:0/2409562033 shutdown_connections 2024-03-29T04:09:57.224 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.243+0000 7f3ce4f82640 1 -- 172.21.15.181:0/2409562033 wait complete. 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.244+0000 7f3ce4f82640 1 Processor -- start 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.244+0000 7f3ce4f82640 1 -- start start 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ce00d0360 0x7f3ce01451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce0152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce006bac0 con 0x7f3ce0072230 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce00b5980 con 0x7f3ce00d0360 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3ce4f82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce00b5630 con 0x7f3ce015c220 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51184/0 (socket says 172.21.15.181:51184) 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 -- 172.21.15.181:0/3828262502 learned_addr learned my addr 172.21.15.181:0/3828262502 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:57.226 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cddd9b640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ce00d0360 0x7f3ce01451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cded9d640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce0152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 -- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 msgr2=0x7f3ce0152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce0152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 -- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ce00d0360 msgr2=0x7f3ce01451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ce00d0360 0x7f3ce01451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.245+0000 7f3cde59c640 1 -- 172.21.15.181:0/3828262502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce0156e40 con 0x7f3ce0072230 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3cde59c640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efc0 tx=0x7f3cd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0018020 con 0x7f3ce0072230 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd0014480 con 0x7f3ce0072230 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce0158d10 con 0x7f3ce0072230 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0014850 con 0x7f3ce0072230 2024-03-29T04:09:57.227 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.246+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01570e0 con 0x7f3ce0072230 2024-03-29T04:09:57.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.247+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0075a10 con 0x7f3ce0072230 2024-03-29T04:09:57.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.247+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3cd0005d10 con 0x7f3ce0072230 2024-03-29T04:09:57.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.248+0000 7f3cceffd640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 0x7f3cc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.248+0000 7f3cddd9b640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 0x7f3cc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.228 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.248+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3cd008c440 con 0x7f3ce0072230 2024-03-29T04:09:57.229 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.248+0000 7f3cddd9b640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 0x7f3cc006e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0158b00 tx=0x7f3cd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.252+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3cd0022070 con 0x7f3ce0072230 2024-03-29T04:09:57.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.428+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ce0159230 con 0x7f3ce0072230 2024-03-29T04:09:57.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.432+0000 7f3cceffd640 1 -- 172.21.15.181:0/3828262502 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7f3cd0010040 con 0x7f3ce0072230 2024-03-29T04:09:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:09:57.412 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 msgr2=0x7f3cc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 0x7f3cc006e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0158b00 tx=0x7f3cd4009040 comp rx=0 tx=0).stop 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 msgr2=0x7f3ce0157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efc0 tx=0x7f3cd000c490 comp rx=0 tx=0).stop 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 shutdown_connections 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cc006c530 0x7f3cc006e9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.435+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ce015c220 0x7f3ce0152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.436+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ce00d0360 0x7f3ce01451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.436+0000 7f3ce4f82640 1 --2- 172.21.15.181:0/3828262502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ce0072230 0x7f3ce0157600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.436+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 >> 172.21.15.181:0/3828262502 conn(0x7f3ce00d0cb0 msgr2=0x7f3ce015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.436+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 shutdown_connections 2024-03-29T04:09:57.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.436+0000 7f3ce4f82640 1 -- 172.21.15.181:0/3828262502 wait complete. 2024-03-29T04:09:57.428 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:09:57.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.552+0000 7fc33cde2640 1 Processor -- start 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.552+0000 7fc33cde2640 1 -- start start 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc33cde2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 0x7fc33815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc33cde2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc3380d1980 0x7fc33815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc33cde2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc33815c990 con 0x7fc338071aa0 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc33cde2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc338071f20 con 0x7fc3380d1980 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc33cde2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc3380ccb70 con 0x7fc3380b7950 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc32ffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc338071aa0 0x7fc3380d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48604/0 (socket says 172.21.15.181:48604) 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.553+0000 7fc32ffff640 1 -- 172.21.15.181:0/3205932772 learned_addr learned my addr 172.21.15.181:0/3205932772 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc33659c640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 0x7fc33815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc336d9d640 1 --2- 172.21.15.181:0/3205932772 >> v2:172.21.15.187:3300/0 conn(0x7fc3380d1980 0x7fc33815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc335d9b640 1 -- 172.21.15.181:0/3205932772 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042328212 0 0) 0x7fc33815c990 con 0x7fc338071aa0 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc335d9b640 1 -- 172.21.15.181:0/3205932772 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc324003520 con 0x7fc338071aa0 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc336d9d640 1 -- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 msgr2=0x7fc33815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc336d9d640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 0x7fc33815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc336d9d640 1 -- 172.21.15.181:0/3205932772 >> v1:172.21.15.181:6789/0 conn(0x7fc338071aa0 legacy=0x7fc3380d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc336d9d640 1 -- 172.21.15.181:0/3205932772 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33815cc00 con 0x7fc3380d1980 2024-03-29T04:09:57.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.554+0000 7fc33659c640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 0x7fc33815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:57.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc336d9d640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380d1980 0x7fc33815fe70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc330009d90 tx=0x7fc33002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db4a16f9cd3003b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc335d9b640 1 -- 172.21.15.181:0/3205932772 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330039070 con 0x7fc3380d1980 2024-03-29T04:09:57.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc335d9b640 1 -- 172.21.15.181:0/3205932772 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33002cc50 con 0x7fc3380d1980 2024-03-29T04:09:57.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc335d9b640 1 -- 172.21.15.181:0/3205932772 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330034440 con 0x7fc3380d1980 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 -- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380d1980 msgr2=0x7fc33815fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380d1980 0x7fc33815fe70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc330009d90 tx=0x7fc33002bea0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 -- 172.21.15.181:0/3205932772 shutdown_connections 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380d1980 0x7fc33815fe70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3205932772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380b7950 0x7fc33815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 -- 172.21.15.181:0/3205932772 >> 172.21.15.181:0/3205932772 conn(0x7fc3380b3cb0 msgr2=0x7fc3380b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 -- 172.21.15.181:0/3205932772 shutdown_connections 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.555+0000 7fc33cde2640 1 -- 172.21.15.181:0/3205932772 wait complete. 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 Processor -- start 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 -- start start 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 0x7fc3380c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380d1980 0x7fc3380cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc338070790 con 0x7fc338071aa0 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3380b7d30 con 0x7fc3380b7950 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.557+0000 7fc33cde2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc3380cc7b0 con 0x7fc3380d1980 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44572/0 (socket says 172.21.15.181:44572) 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc32ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 0x7fc3380c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 -- 172.21.15.181:0/3364296798 learned_addr learned my addr 172.21.15.181:0/3364296798 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc336d9d640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380d1980 0x7fc3380cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 -- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380d1980 msgr2=0x7fc3380cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380d1980 0x7fc3380cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 -- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 msgr2=0x7fc3380c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 0x7fc3380c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 -- 172.21.15.181:0/3364296798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3380c9ba0 con 0x7fc338071aa0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc32ffff640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 0x7fc3380c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33659c640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ec10 tx=0x7fc32800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328017070 con 0x7fc338071aa0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32800cc30 con 0x7fc338071aa0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328013470 con 0x7fc338071aa0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3380c9e80 con 0x7fc338071aa0 2024-03-29T04:09:57.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.558+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3380ca440 con 0x7fc338071aa0 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.559+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc338075890 con 0x7fc338071aa0 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.560+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc32801a070 con 0x7fc338071aa0 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.560+0000 7fc32effd640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 0x7fc31806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.560+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc32808bcf0 con 0x7fc338071aa0 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.560+0000 7fc32ffff640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 0x7fc31806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.560+0000 7fc32ffff640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 0x7fc31806e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3200011e0 tx=0x7fc320007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.564+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc328090070 con 0x7fc338071aa0 2024-03-29T04:09:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.739+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc338070320 con 0x7fc338071aa0 2024-03-29T04:09:57.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.740+0000 7fc32effd640 1 -- 172.21.15.181:0/3364296798 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc3280598c0 con 0x7fc338071aa0 2024-03-29T04:09:57.720 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 msgr2=0x7fc31806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 0x7fc31806e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3200011e0 tx=0x7fc320007d30 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 msgr2=0x7fc3381f1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ec10 tx=0x7fc32800c520 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 shutdown_connections 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc31806c530 0x7fc31806e9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3380d1980 0x7fc3380cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3380b7950 0x7fc3380c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 --2- 172.21.15.181:0/3364296798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc338071aa0 0x7fc3381f1670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 >> 172.21.15.181:0/3364296798 conn(0x7fc3380b3cb0 msgr2=0x7fc3380b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 shutdown_connections 2024-03-29T04:09:57.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.742+0000 7fc33cde2640 1 -- 172.21.15.181:0/3364296798 wait complete. 2024-03-29T04:09:57.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:09:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.858+0000 7f20fe29a640 1 Processor -- start 2024-03-29T04:09:57.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.858+0000 7f20fe29a640 1 -- start start 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fe29a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20f8072d50 0x7f20f80b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fe29a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f815fc70 con 0x7f20f815f800 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fe29a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f80b35e0 con 0x7f20f8144cb0 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fe29a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20f80b3e80 con 0x7f20f8072d50 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fc837640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f20f815f800 0x7f20f80ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48620/0 (socket says 172.21.15.181:48620) 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20fc837640 1 -- 172.21.15.181:0/991865344 learned_addr learned my addr 172.21.15.181:0/991865344 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f77fe640 1 --2- 172.21.15.181:0/991865344 >> v2:172.21.15.187:3300/0 conn(0x7f20f8072d50 0x7f20f80b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154410172 0 0) 0x7f20f815fc70 con 0x7f20f815f800 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20e0003520 con 0x7f20f815f800 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467693731 0 0) 0x7f20f80b35e0 con 0x7f20f8144cb0 2024-03-29T04:09:57.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20f815fc70 con 0x7f20f8144cb0 2024-03-29T04:09:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f77fe640 1 -- 172.21.15.181:0/991865344 >> v1:172.21.15.187:6789/0 conn(0x7f20f8144cb0 legacy=0x7f20f8072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f77fe640 1 -- 172.21.15.181:0/991865344 >> v1:172.21.15.181:6789/0 conn(0x7f20f815f800 legacy=0x7f20f80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.859+0000 7f20f77fe640 1 -- 172.21.15.181:0/991865344 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f80b38c0 con 0x7f20f8072d50 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.860+0000 7f20f77fe640 1 --2- 172.21.15.181:0/991865344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f80b2f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20f80b4000 tx=0x7f20ec02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c4c0a46573ed39f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.860+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec034070 con 0x7f20f8072d50 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.860+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec030610 con 0x7f20f8072d50 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.860+0000 7f20f6ffd640 1 -- 172.21.15.181:0/991865344 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0309e0 con 0x7f20f8072d50 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 -- 172.21.15.181:0/991865344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 msgr2=0x7f20f80b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 --2- 172.21.15.181:0/991865344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f80b2f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20f80b4000 tx=0x7f20ec02b960 comp rx=0 tx=0).stop 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 -- 172.21.15.181:0/991865344 shutdown_connections 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 --2- 172.21.15.181:0/991865344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f80b2f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 -- 172.21.15.181:0/991865344 >> 172.21.15.181:0/991865344 conn(0x7f20f80b5b00 msgr2=0x7f20f806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 -- 172.21.15.181:0/991865344 shutdown_connections 2024-03-29T04:09:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.861+0000 7f20fe29a640 1 -- 172.21.15.181:0/991865344 wait complete. 2024-03-29T04:09:57.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.862+0000 7f20fe29a640 1 Processor -- start 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.862+0000 7f20fe29a640 1 -- start start 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20f8144cb0 0x7f20f80cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80ce760 con 0x7f20f815f800 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80cdb30 con 0x7f20f8072d50 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20f80ce8e0 con 0x7f20f8144cb0 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44578/0 (socket says 172.21.15.181:44578) 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 -- 172.21.15.181:0/2777932319 learned_addr learned my addr 172.21.15.181:0/2777932319 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:57.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20f7fff640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20f77fe640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20f8144cb0 0x7f20f80cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 -- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20f8144cb0 msgr2=0x7f20f80cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20f8144cb0 0x7f20f80cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 -- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 msgr2=0x7f20f815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 -- 172.21.15.181:0/2777932319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f80cf4c0 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fc837640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20f000caf0 tx=0x7f20f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0017070 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f000aca0 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f80cf7a0 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0012600 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.863+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f8142440 con 0x7f20f815f800 2024-03-29T04:09:57.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.864+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8142640 con 0x7f20f815f800 2024-03-29T04:09:57.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.865+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20f001a070 con 0x7f20f815f800 2024-03-29T04:09:57.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.865+0000 7f20f4ff9640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 0x7f20e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:57.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.865+0000 7f20f7fff640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 0x7f20e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:57.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.865+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f20f008b7a0 con 0x7f20f815f800 2024-03-29T04:09:57.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.866+0000 7f20f7fff640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 0x7f20e406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20e8007fd0 tx=0x7f20e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:57.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:57.870+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20f8142640 con 0x7f20f815f800 2024-03-29T04:09:58.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.043+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20f8075890 con 0x7f20f815f800 2024-03-29T04:09:58.023 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.044+0000 7f20f4ff9640 1 -- 172.21.15.181:0/2777932319 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f20f00592f0 con 0x7f20f815f800 2024-03-29T04:09:58.024 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:58.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 msgr2=0x7f20e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 0x7f20e406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20e8007fd0 tx=0x7f20e8009040 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 msgr2=0x7f20f80aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20f000caf0 tx=0x7f20f000a060 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 shutdown_connections 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20f815f800 0x7f20f80aed90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f20e406c530 0x7f20e406e9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20f8144cb0 0x7f20f80cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 --2- 172.21.15.181:0/2777932319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f8072d50 0x7f20f815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 >> 172.21.15.181:0/2777932319 conn(0x7f20f80b5b00 msgr2=0x7f20f806d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 shutdown_connections 2024-03-29T04:09:58.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.046+0000 7f20fe29a640 1 -- 172.21.15.181:0/2777932319 wait complete. 2024-03-29T04:09:58.036 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:09:58.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.160+0000 7ff4b0924640 1 Processor -- start 2024-03-29T04:09:58.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.160+0000 7ff4b0924640 1 -- start start 2024-03-29T04:09:58.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4b0924640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 0x7ff4ac06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4b0924640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ac06d720 0x7ff4ac15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4b0924640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0ce9c0 con 0x7ff4ac1603d0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4b0924640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac160e30 con 0x7ff4ac06d720 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4b0924640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac160b50 con 0x7ff4ac13f0c0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ac06d720 0x7ff4ac15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4ac06d720 0x7ff4ac15fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44580/0 (socket says 172.21.15.181:44580) 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 -- 172.21.15.181:0/2629560122 learned_addr learned my addr 172.21.15.181:0/2629560122 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4aa59c640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 0x7ff4ac06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a959a640 1 -- 172.21.15.181:0/2629560122 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621275780 0 0) 0x7ff49c001110 con 0x7ff4ac1603d0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a959a640 1 -- 172.21.15.181:0/2629560122 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff494003520 con 0x7ff4ac1603d0 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 -- 172.21.15.181:0/2629560122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 msgr2=0x7ff4ac06d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 0x7ff4ac06d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 -- 172.21.15.181:0/2629560122 >> v1:172.21.15.187:6789/0 conn(0x7ff4ac1603d0 legacy=0x7ff4ac0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4a9d9b640 1 -- 172.21.15.181:0/2629560122 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac13fa40 con 0x7ff4ac06d720 2024-03-29T04:09:58.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.161+0000 7ff4aa59c640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 0x7ff4ac06d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:09:58.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.162+0000 7ff4a9d9b640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac15fe90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009870 tx=0x7ff4a002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7563793fd5ce8d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.162+0000 7ff4a959a640 1 -- 172.21.15.181:0/2629560122 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00317c0 con 0x7ff4ac06d720 2024-03-29T04:09:58.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.162+0000 7ff4a959a640 1 -- 172.21.15.181:0/2629560122 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0037070 con 0x7ff4ac06d720 2024-03-29T04:09:58.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.162+0000 7ff4a959a640 1 -- 172.21.15.181:0/2629560122 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0031b60 con 0x7ff4ac06d720 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 -- 172.21.15.181:0/2629560122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 msgr2=0x7ff4ac15fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac15fe90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009870 tx=0x7ff4a002b860 comp rx=0 tx=0).stop 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 -- 172.21.15.181:0/2629560122 shutdown_connections 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac15fe90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 --2- 172.21.15.181:0/2629560122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac13f0c0 0x7ff4ac06d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 -- 172.21.15.181:0/2629560122 >> 172.21.15.181:0/2629560122 conn(0x7ff4ac0b5b00 msgr2=0x7ff4ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 -- 172.21.15.181:0/2629560122 shutdown_connections 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.163+0000 7ff4b0924640 1 -- 172.21.15.181:0/2629560122 wait complete. 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.164+0000 7ff4b0924640 1 Processor -- start 2024-03-29T04:09:58.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.164+0000 7ff4b0924640 1 -- start start 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ac13f0c0 0x7ff4ac072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac1603d0 0x7ff4ac0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac160820 con 0x7ff4ac06d720 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac140000 con 0x7ff4ac13f0c0 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4b0924640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac13f4a0 con 0x7ff4ac1603d0 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44592/0 (socket says 172.21.15.181:44592) 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 -- 172.21.15.181:0/3599349484 learned_addr learned my addr 172.21.15.181:0/3599349484 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4a9d9b640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ac13f0c0 0x7ff4ac072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aad9d640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac1603d0 0x7ff4ac0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 -- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac1603d0 msgr2=0x7ff4ac0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac1603d0 0x7ff4ac0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 -- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ac13f0c0 msgr2=0x7ff4ac072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ac13f0c0 0x7ff4ac072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.165+0000 7ff4aa59c640 1 -- 172.21.15.181:0/3599349484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0b3600 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff4aa59c640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400efc0 tx=0x7ff4a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4018020 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a4014680 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac0b38e0 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4014a50 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.166+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac0b3ea0 con 0x7ff4ac06d720 2024-03-29T04:09:58.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.167+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac075890 con 0x7ff4ac06d720 2024-03-29T04:09:58.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.167+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff4a4005d10 con 0x7ff4ac06d720 2024-03-29T04:09:58.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.168+0000 7ff49affd640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 0x7ff48c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.168+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff4a408c710 con 0x7ff4ac06d720 2024-03-29T04:09:58.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.168+0000 7ff4a9d9b640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 0x7ff48c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.168+0000 7ff4a9d9b640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 0x7ff48c06e9f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0036000 tx=0x7ff4a003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.172+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff4a4022030 con 0x7ff4ac06d720 2024-03-29T04:09:58.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.344+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4ac06c2c0 con 0x7ff4ac06d720 2024-03-29T04:09:58.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.344+0000 7ff49affd640 1 -- 172.21.15.181:0/3599349484 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ff4a4010040 con 0x7ff4ac06d720 2024-03-29T04:09:58.326 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 msgr2=0x7ff48c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 0x7ff48c06e9f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0036000 tx=0x7ff4a003d040 comp rx=0 tx=0).stop 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 msgr2=0x7ff4ac0b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400efc0 tx=0x7ff4a400c490 comp rx=0 tx=0).stop 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 shutdown_connections 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff48c06c530 0x7ff48c06e9f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4ac1603d0 0x7ff4ac0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4ac13f0c0 0x7ff4ac072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 --2- 172.21.15.181:0/3599349484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4ac06d720 0x7ff4ac0b3020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 >> 172.21.15.181:0/3599349484 conn(0x7ff4ac0b5b00 msgr2=0x7ff4ac0d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 shutdown_connections 2024-03-29T04:09:58.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.346+0000 7ff4b0924640 1 -- 172.21.15.181:0/3599349484 wait complete. 2024-03-29T04:09:58.337 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:09:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.462+0000 7fcee6a0d640 1 Processor -- start 2024-03-29T04:09:58.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.462+0000 7fcee6a0d640 1 -- start start 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcee6a0d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcee00ae850 0x7fcee00b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcee6a0d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee00cfba0 con 0x7fcee015e840 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcee6a0d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee00cfe60 con 0x7fcee00d10b0 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcee6a0d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcee00d0140 con 0x7fcee00ae850 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcedffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcee015e840 0x7fcee00b0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48630/0 (socket says 172.21.15.181:48630) 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcedffff640 1 -- 172.21.15.181:0/1537661821 learned_addr learned my addr 172.21.15.181:0/1537661821 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcee4faa640 1 --2- 172.21.15.181:0/1537661821 >> v2:172.21.15.187:3300/0 conn(0x7fcee00ae850 0x7fcee00b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.463+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262075595 0 0) 0x7fcee00cfba0 con 0x7fcee015e840 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcec8003520 con 0x7fcee015e840 2024-03-29T04:09:58.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287798638 0 0) 0x7fcee00cfe60 con 0x7fcee00d10b0 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcee00cfba0 con 0x7fcee00d10b0 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcee4faa640 1 -- 172.21.15.181:0/1537661821 >> v1:172.21.15.187:6789/0 conn(0x7fcee00d10b0 legacy=0x7fcee00ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcee4faa640 1 -- 172.21.15.181:0/1537661821 >> v1:172.21.15.181:6789/0 conn(0x7fcee015e840 legacy=0x7fcee00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcee4faa640 1 -- 172.21.15.181:0/1537661821 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee00af960 con 0x7fcee00ae850 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.464+0000 7fcee4faa640 1 --2- 172.21.15.181:0/1537661821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee00b0870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced8009d90 tx=0x7fced802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d3a38e65df41e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8039070 con 0x7fcee00ae850 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced802cc60 con 0x7fcee00ae850 2024-03-29T04:09:58.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcedeffd640 1 -- 172.21.15.181:0/1537661821 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8034660 con 0x7fcee00ae850 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 -- 172.21.15.181:0/1537661821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 msgr2=0x7fcee00b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/1537661821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee00b0870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fced8009d90 tx=0x7fced802beb0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 -- 172.21.15.181:0/1537661821 shutdown_connections 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/1537661821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee00b0870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 -- 172.21.15.181:0/1537661821 >> 172.21.15.181:0/1537661821 conn(0x7fcee00d0380 msgr2=0x7fcee00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 -- 172.21.15.181:0/1537661821 shutdown_connections 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.465+0000 7fcee6a0d640 1 -- 172.21.15.181:0/1537661821 wait complete. 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.466+0000 7fcee6a0d640 1 Processor -- start 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.466+0000 7fcee6a0d640 1 -- start start 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee0099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee00d10b0 0x7fcee0161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0068e90 con 0x7fcee015e840 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcee015ec20 con 0x7fcee00ae850 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee6a0d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcee0069010 con 0x7fcee00d10b0 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44600/0 (socket says 172.21.15.181:44600) 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 -- 172.21.15.181:0/2994599934 learned_addr learned my addr 172.21.15.181:0/2994599934 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcedffff640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee0099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcedf7fe640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee00d10b0 0x7fcee0161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 -- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee00d10b0 msgr2=0x7fcee0161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee00d10b0 0x7fcee0161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.467+0000 7fcee4faa640 1 -- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 msgr2=0x7fcee0099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcee4faa640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee0099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcee4faa640 1 -- 172.21.15.181:0/2994599934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee01604e0 con 0x7fcee015e840 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcee4faa640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fced8009d90 tx=0x7fced8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8039050 con 0x7fcee015e840 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8045070 con 0x7fcee015e840 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee01607c0 con 0x7fcee015e840 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8009450 con 0x7fcee015e840 2024-03-29T04:09:58.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.468+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee0160d80 con 0x7fcee015e840 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.469+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee0075890 con 0x7fcee015e840 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.469+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fced803e850 con 0x7fcee015e840 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.470+0000 7fcedcff9640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 0x7fcecc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.470+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fced80ae730 con 0x7fcee015e840 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.470+0000 7fcedffff640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 0x7fcecc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.470+0000 7fcedffff640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 0x7fcecc06e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fced0000f60 tx=0x7fced0007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.474+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fced807c4b0 con 0x7fcee015e840 2024-03-29T04:09:58.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.649+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcee006e960 con 0x7fcee015e840 2024-03-29T04:09:58.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.649+0000 7fcedcff9640 1 -- 172.21.15.181:0/2994599934 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fced80807b0 con 0x7fcee015e840 2024-03-29T04:09:58.629 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 msgr2=0x7fcecc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 0x7fcecc06e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fced0000f60 tx=0x7fced0007d70 comp rx=0 tx=0).stop 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 msgr2=0x7fcee00c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fced8009d90 tx=0x7fced8004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 shutdown_connections 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee015e840 0x7fcee00c8490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcecc06c530 0x7fcecc06e9f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcee00d10b0 0x7fcee0161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 --2- 172.21.15.181:0/2994599934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcee00ae850 0x7fcee0099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 >> 172.21.15.181:0/2994599934 conn(0x7fcee00d0380 msgr2=0x7fcee006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 shutdown_connections 2024-03-29T04:09:58.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.651+0000 7fcee6a0d640 1 -- 172.21.15.181:0/2994599934 wait complete. 2024-03-29T04:09:58.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:09:58.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.764+0000 7f8373463640 1 Processor -- start 2024-03-29T04:09:58.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.764+0000 7f8373463640 1 -- start start 2024-03-29T04:09:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f8373463640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f836c072630 0x7f836c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f8373463640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c06b580 con 0x7f836c0721b0 2024-03-29T04:09:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f8373463640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c15c9c0 con 0x7f836c06c070 2024-03-29T04:09:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f8373463640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f836c15d780 con 0x7f836c072630 2024-03-29T04:09:58.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f83709fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f836c0721b0 0x7f836c15fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48636/0 (socket says 172.21.15.181:48636) 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f83709fe640 1 -- 172.21.15.181:0/1906527111 learned_addr learned my addr 172.21.15.181:0/1906527111 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.765+0000 7f8371a00640 1 --2- 172.21.15.181:0/1906527111 >> v2:172.21.15.187:3300/0 conn(0x7f836c072630 0x7f836c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539325060 0 0) 0x7f836c06b580 con 0x7f836c0721b0 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8358003520 con 0x7f836c0721b0 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797586430 0 0) 0x7f836c15c9c0 con 0x7f836c06c070 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f836c06b580 con 0x7f836c06c070 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8371a00640 1 -- 172.21.15.181:0/1906527111 >> v1:172.21.15.187:6789/0 conn(0x7f836c06c070 legacy=0x7f836c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8371a00640 1 -- 172.21.15.181:0/1906527111 >> v1:172.21.15.181:6789/0 conn(0x7f836c0721b0 legacy=0x7f836c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8371a00640 1 -- 172.21.15.181:0/1906527111 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c15cca0 con 0x7f836c072630 2024-03-29T04:09:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.766+0000 7f8371a00640 1 --2- 172.21.15.181:0/1906527111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c072630 0x7f836c1604b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8364009d90 tx=0x7f836402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c9726fcdccc783 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364039070 con 0x7f836c072630 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836402cd00 con 0x7f836c072630 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8363fff640 1 -- 172.21.15.181:0/1906527111 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364034640 con 0x7f836c072630 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 -- 172.21.15.181:0/1906527111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c072630 msgr2=0x7f836c1604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 --2- 172.21.15.181:0/1906527111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c072630 0x7f836c1604b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8364009d90 tx=0x7f836402bf50 comp rx=0 tx=0).stop 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 -- 172.21.15.181:0/1906527111 shutdown_connections 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 --2- 172.21.15.181:0/1906527111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c072630 0x7f836c1604b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 -- 172.21.15.181:0/1906527111 >> 172.21.15.181:0/1906527111 conn(0x7f836c0d0cb0 msgr2=0x7f836c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 -- 172.21.15.181:0/1906527111 shutdown_connections 2024-03-29T04:09:58.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.767+0000 7f8373463640 1 -- 172.21.15.181:0/1906527111 wait complete. 2024-03-29T04:09:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 Processor -- start 2024-03-29T04:09:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 -- start start 2024-03-29T04:09:58.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06c070 0x7f836c1433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0721b0 0x7f836c1e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f836c15d260 con 0x7f836c072630 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f836c156cc0 con 0x7f836c06c070 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8373463640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f836c0ccd60 con 0x7f836c0721b0 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8371a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8371a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44606/0 (socket says 172.21.15.181:44606) 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f8371a00640 1 -- 172.21.15.181:0/3059450506 learned_addr learned my addr 172.21.15.181:0/3059450506 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f83709fe640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0721b0 0x7f836c1e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.769+0000 7f83711ff640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06c070 0x7f836c1433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 -- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0721b0 msgr2=0x7f836c1e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0721b0 0x7f836c1e57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 -- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06c070 msgr2=0x7f836c1433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06c070 0x7f836c1433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:58.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 -- 172.21.15.181:0/3059450506 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c143920 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8371a00640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8364009d90 tx=0x7f8364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364039050 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8364045070 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c1e61b0 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364009450 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.770+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c1e6770 con 0x7f836c072630 2024-03-29T04:09:58.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.771+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836c075a10 con 0x7f836c072630 2024-03-29T04:09:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.772+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f836403e830 con 0x7f836c072630 2024-03-29T04:09:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.772+0000 7f8361ffb640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 0x7f835006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.772+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f83640adf80 con 0x7f836c072630 2024-03-29T04:09:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.772+0000 7f83711ff640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 0x7f835006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:58.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.773+0000 7f83711ff640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 0x7f835006ea20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8368007fd0 tx=0x7f8368009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:58.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:58.777+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f836404e020 con 0x7f836c072630 2024-03-29T04:09:58.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.020+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f836c0b4160 con 0x7f836c072630 2024-03-29T04:09:59.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.439+0000 7f8361ffb640 1 -- 172.21.15.181:0/3059450506 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v66) ==== 150+0+0 (secure 0 0 0) 0x7f836c143920 con 0x7f836c072630 2024-03-29T04:09:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.441+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 msgr2=0x7f835006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.441+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 0x7f835006ea20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8368007fd0 tx=0x7f8368009040 comp rx=0 tx=0).stop 2024-03-29T04:09:59.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 msgr2=0x7f836c1f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8364009d90 tx=0x7f8364004a40 comp rx=0 tx=0).stop 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 shutdown_connections 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f836c072630 0x7f836c1f5470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f835006c560 0x7f835006ea20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0721b0 0x7f836c1e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 --2- 172.21.15.181:0/3059450506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06c070 0x7f836c1433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 >> 172.21.15.181:0/3059450506 conn(0x7f836c0d0cb0 msgr2=0x7f836c0ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 shutdown_connections 2024-03-29T04:09:59.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.442+0000 7f8373463640 1 -- 172.21.15.181:0/3059450506 wait complete. 2024-03-29T04:09:59.433 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T04:09:59.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 Processor -- start 2024-03-29T04:09:59.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 -- start start 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1d4073310 0x7fd1d406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d4073880 con 0x7fd1d415e5b0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d40d0e20 con 0x7fd1d40d1be0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.557+0000 7fd1da959640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d40d1100 con 0x7fd1d4073310 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d3fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd1d415e5b0 0x7fd1d4162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48640/0 (socket says 172.21.15.181:48640) 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d3fff640 1 -- 172.21.15.181:0/1365964813 learned_addr learned my addr 172.21.15.181:0/1365964813 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d37fe640 1 --2- 172.21.15.181:0/1365964813 >> v2:172.21.15.187:3300/0 conn(0x7fd1d4073310 0x7fd1d406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579746959 0 0) 0x7fd1d4073880 con 0x7fd1d415e5b0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1bc003520 con 0x7fd1d415e5b0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437167612 0 0) 0x7fd1d40d0e20 con 0x7fd1d40d1be0 2024-03-29T04:09:59.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1d4073880 con 0x7fd1d40d1be0 2024-03-29T04:09:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d37fe640 1 -- 172.21.15.181:0/1365964813 >> v1:172.21.15.187:6789/0 conn(0x7fd1d40d1be0 legacy=0x7fd1d406f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d37fe640 1 -- 172.21.15.181:0/1365964813 >> v1:172.21.15.181:6789/0 conn(0x7fd1d415e5b0 legacy=0x7fd1d4162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.558+0000 7fd1d37fe640 1 -- 172.21.15.181:0/1365964813 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d40d13e0 con 0x7fd1d4073310 2024-03-29T04:09:59.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1d37fe640 1 --2- 172.21.15.181:0/1365964813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d4073310 0x7fd1d406b580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097a0 tx=0x7fd1c802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4870db06b0b4b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035020 con 0x7fd1d4073310 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c8031830 con 0x7fd1d4073310 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1d2ffd640 1 -- 172.21.15.181:0/1365964813 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8031be0 con 0x7fd1d4073310 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 -- 172.21.15.181:0/1365964813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d4073310 msgr2=0x7fd1d406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 --2- 172.21.15.181:0/1365964813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d4073310 0x7fd1d406b580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097a0 tx=0x7fd1c802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 -- 172.21.15.181:0/1365964813 shutdown_connections 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 --2- 172.21.15.181:0/1365964813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d4073310 0x7fd1d406b580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 -- 172.21.15.181:0/1365964813 >> 172.21.15.181:0/1365964813 conn(0x7fd1d406e720 msgr2=0x7fd1d406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:59.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 -- 172.21.15.181:0/1365964813 shutdown_connections 2024-03-29T04:09:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.559+0000 7fd1da959640 1 -- 172.21.15.181:0/1365964813 wait complete. 2024-03-29T04:09:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.560+0000 7fd1da959640 1 Processor -- start 2024-03-29T04:09:59.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 -- start start 2024-03-29T04:09:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d4073310 0x7fd1d4159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d415e5b0 0x7fd1d415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d406bac0 con 0x7fd1d40d1be0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d415e990 con 0x7fd1d415e5b0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1da959640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d406bc40 con 0x7fd1d4073310 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1d37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.561+0000 7fd1d37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44610/0 (socket says 172.21.15.181:44610) 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 -- 172.21.15.181:0/2362900700 learned_addr learned my addr 172.21.15.181:0/2362900700 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d3fff640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d4073310 0x7fd1d4159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d8ef6640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d415e5b0 0x7fd1d415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 -- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d4073310 msgr2=0x7fd1d4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d4073310 0x7fd1d4159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 -- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d415e5b0 msgr2=0x7fd1d415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d415e5b0 0x7fd1d415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 -- 172.21.15.181:0/2362900700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d415a6e0 con 0x7fd1d40d1be0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d37fe640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80098c0 tx=0x7fd1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035040 con 0x7fd1d40d1be0 2024-03-29T04:09:59.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c803f070 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d40b3db0 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c803bb60 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.562+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d4158a80 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.563+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d4075890 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.563+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd1c8038020 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.564+0000 7fd1d0ff9640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 0x7fd1c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.564+0000 7fd1d3fff640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 0x7fd1c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.564+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd1c80ad5b0 con 0x7fd1d40d1be0 2024-03-29T04:09:59.545 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.564+0000 7fd1d3fff640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 0x7fd1c006e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000f60 tx=0x7fd1c4007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:09:59.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.568+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd1c807b190 con 0x7fd1d40d1be0 2024-03-29T04:09:59.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.746+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd1d4072be0 con 0x7fd1d40d1be0 2024-03-29T04:09:59.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.752+0000 7fd1d0ff9640 1 -- 172.21.15.181:0/2362900700 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fd1c807f490 con 0x7fd1d40d1be0 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 msgr2=0x7fd1c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 0x7fd1c006e9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000f60 tx=0x7fd1c4007d70 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 msgr2=0x7fd1d415a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80098c0 tx=0x7fd1c80046c0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 shutdown_connections 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1d415e5b0 0x7fd1d415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d40d1be0 0x7fd1d415a0b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1c006c530 0x7fd1c006e9f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 --2- 172.21.15.181:0/2362900700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1d4073310 0x7fd1d4159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 >> 172.21.15.181:0/2362900700 conn(0x7fd1d406e720 msgr2=0x7fd1d41e75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 shutdown_connections 2024-03-29T04:09:59.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:09:59.754+0000 7fd1da959640 1 -- 172.21.15.181:0/2362900700 wait complete. 2024-03-29T04:09:59.746 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:09:59.746 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:09:59.792 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:09:59.830 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:09:59.830 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:59.830 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:59.846 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:59.846 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:59.902 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:09:59.902 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:09:59.958 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:00.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.121+0000 7fe88e13d640 1 Processor -- start 2024-03-29T04:10:00.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.121+0000 7fe88e13d640 1 -- start start 2024-03-29T04:10:00.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe88e13d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe88815e950 0x7fe8880d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe88e13d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8880d36f0 con 0x7fe8880d30d0 2024-03-29T04:10:00.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe88e13d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8880d39d0 con 0x7fe8880d48c0 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe88e13d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe8880d3cb0 con 0x7fe88815e950 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe887fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe8880d30d0 0x7fe8880b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48654/0 (socket says 172.21.15.181:48654) 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe887fff640 1 -- 172.21.15.181:0/855544446 learned_addr learned my addr 172.21.15.181:0/855544446 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.122+0000 7fe8877fe640 1 --2- 172.21.15.181:0/855544446 >> v2:172.21.15.187:3300/0 conn(0x7fe88815e950 0x7fe8880d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.123+0000 7fe8867fc640 1 -- 172.21.15.181:0/855544446 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528123885 0 0) 0x7fe8880d36f0 con 0x7fe8880d30d0 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.123+0000 7fe8867fc640 1 -- 172.21.15.181:0/855544446 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe870003520 con 0x7fe8880d30d0 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.123+0000 7fe8877fe640 1 -- 172.21.15.181:0/855544446 >> v1:172.21.15.187:6789/0 conn(0x7fe8880d48c0 legacy=0x7fe8880d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.123+0000 7fe8877fe640 1 -- 172.21.15.181:0/855544446 >> v1:172.21.15.181:6789/0 conn(0x7fe8880d30d0 legacy=0x7fe8880b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.123+0000 7fe8877fe640 1 -- 172.21.15.181:0/855544446 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8880b7ac0 con 0x7fe88815e950 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe8877fe640 1 --2- 172.21.15.181:0/855544446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880d4380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8780097a0 tx=0x7fe87802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43545a43a0bc308 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe8867fc640 1 -- 172.21.15.181:0/855544446 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87802f810 con 0x7fe88815e950 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe8867fc640 1 -- 172.21.15.181:0/855544446 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87802f9b0 con 0x7fe88815e950 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe8867fc640 1 -- 172.21.15.181:0/855544446 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87802fd60 con 0x7fe88815e950 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 -- 172.21.15.181:0/855544446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 msgr2=0x7fe8880d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 --2- 172.21.15.181:0/855544446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880d4380 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8780097a0 tx=0x7fe87802b9f0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 -- 172.21.15.181:0/855544446 shutdown_connections 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 --2- 172.21.15.181:0/855544446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880d4380 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 -- 172.21.15.181:0/855544446 >> 172.21.15.181:0/855544446 conn(0x7fe8880676e0 msgr2=0x7fe888067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 -- 172.21.15.181:0/855544446 shutdown_connections 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.124+0000 7fe88e13d640 1 -- 172.21.15.181:0/855544446 wait complete. 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.125+0000 7fe88e13d640 1 Processor -- start 2024-03-29T04:10:00.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 -- start start 2024-03-29T04:10:00.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8880d48c0 0x7fe8881589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe88815ed30 con 0x7fe8880d30d0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8880d4d00 con 0x7fe88815e950 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe88e13d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe88815eeb0 con 0x7fe8880d48c0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe8877fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe8877fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44614/0 (socket says 172.21.15.181:44614) 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.126+0000 7fe8877fe640 1 -- 172.21.15.181:0/1084877046 learned_addr learned my addr 172.21.15.181:0/1084877046 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe887fff640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe886ffd640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8880d48c0 0x7fe8881589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 -- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8880d48c0 msgr2=0x7fe8881589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8880d48c0 0x7fe8881589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 -- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 msgr2=0x7fe8880cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 -- 172.21.15.181:0/1084877046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe888158ef0 con 0x7fe8880d30d0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe8877fe640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8780097a0 tx=0x7fe8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878047070 con 0x7fe8880d30d0 2024-03-29T04:10:00.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe878004230 con 0x7fe8880d30d0 2024-03-29T04:10:00.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8881482b0 con 0x7fe8880d30d0 2024-03-29T04:10:00.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878037610 con 0x7fe8880d30d0 2024-03-29T04:10:00.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.127+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe888148870 con 0x7fe8880d30d0 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.128+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888075a10 con 0x7fe8880d30d0 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.129+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe87804a070 con 0x7fe8880d30d0 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.129+0000 7fe86ffff640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 0x7fe87406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.129+0000 7fe886ffd640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 0x7fe87406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.129+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe8780ad340 con 0x7fe8880d30d0 2024-03-29T04:10:00.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.130+0000 7fe886ffd640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 0x7fe87406e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe8881f75e0 tx=0x7fe87c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.113 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.134+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe878033030 con 0x7fe8880d30d0 2024-03-29T04:10:00.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.310+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe88806dc10 con 0x7fe8880d30d0 2024-03-29T04:10:00.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.311+0000 7fe86ffff640 1 -- 172.21.15.181:0/1084877046 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7fe87807afa0 con 0x7fe8880d30d0 2024-03-29T04:10:00.291 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:00.292 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 msgr2=0x7fe87406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 0x7fe87406e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe8881f75e0 tx=0x7fe87c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 msgr2=0x7fe8880cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8780097a0 tx=0x7fe8780046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 shutdown_connections 2024-03-29T04:10:00.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe88815e950 0x7fe8880cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe87406c530 0x7fe87406e9f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8880d48c0 0x7fe8881589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 --2- 172.21.15.181:0/1084877046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8880d30d0 0x7fe8880cd8c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.313+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 >> 172.21.15.181:0/1084877046 conn(0x7fe8880676e0 msgr2=0x7fe88815d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.314+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 shutdown_connections 2024-03-29T04:10:00.294 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.314+0000 7fe88e13d640 1 -- 172.21.15.181:0/1084877046 wait complete. 2024-03-29T04:10:00.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:00.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 Processor -- start 2024-03-29T04:10:00.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 -- start start 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7600b5dc0 0x7fb7600b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7600b73d0 0x7fb7600b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600b66e0 con 0x7fb7600ba090 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600d2d90 con 0x7fb7600b5dc0 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb766159640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600d2ff0 con 0x7fb7600b73d0 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.431+0000 7fb75f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7600b5dc0 0x7fb7600b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7600b5dc0 0x7fb7600b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44618/0 (socket says 172.21.15.181:44618) 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 -- 172.21.15.181:0/3388921377 learned_addr learned my addr 172.21.15.181:0/3388921377 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75ffff640 1 --2- 172.21.15.181:0/3388921377 >> v2:172.21.15.187:3300/0 conn(0x7fb7600b73d0 0x7fb7600b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 -- 172.21.15.181:0/3388921377 >> v2:172.21.15.187:3300/0 conn(0x7fb7600b73d0 msgr2=0x7fb7600b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 --2- 172.21.15.181:0/3388921377 >> v2:172.21.15.187:3300/0 conn(0x7fb7600b73d0 0x7fb7600b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 -- 172.21.15.181:0/3388921377 >> v1:172.21.15.187:6789/0 conn(0x7fb7600ba090 legacy=0x7fb7600d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 -- 172.21.15.181:0/3388921377 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7600d32d0 con 0x7fb7600b5dc0 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.432+0000 7fb75f7fe640 1 --2- 172.21.15.181:0/3388921377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7600b61a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7600d5420 tx=0x7fb75002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c10560118bc0fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb75e7fc640 1 -- 172.21.15.181:0/3388921377 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750034070 con 0x7fb7600b5dc0 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb75e7fc640 1 -- 172.21.15.181:0/3388921377 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb750030860 con 0x7fb7600b5dc0 2024-03-29T04:10:00.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb75e7fc640 1 -- 172.21.15.181:0/3388921377 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750030c30 con 0x7fb7600b5dc0 2024-03-29T04:10:00.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 -- 172.21.15.181:0/3388921377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 msgr2=0x7fb7600b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 --2- 172.21.15.181:0/3388921377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7600b61a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb7600d5420 tx=0x7fb75002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 -- 172.21.15.181:0/3388921377 shutdown_connections 2024-03-29T04:10:00.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 --2- 172.21.15.181:0/3388921377 >> v2:172.21.15.187:3300/0 conn(0x7fb7600b73d0 0x7fb7600b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 --2- 172.21.15.181:0/3388921377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7600b61a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 -- 172.21.15.181:0/3388921377 >> 172.21.15.181:0/3388921377 conn(0x7fb7600678f0 msgr2=0x7fb7600d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 -- 172.21.15.181:0/3388921377 shutdown_connections 2024-03-29T04:10:00.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.433+0000 7fb766159640 1 -- 172.21.15.181:0/3388921377 wait complete. 2024-03-29T04:10:00.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.434+0000 7fb766159640 1 Processor -- start 2024-03-29T04:10:00.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 -- start start 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7600b73d0 0x7fb760160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7600ba090 0x7fb76015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600d3b70 con 0x7fb7600b5dc0 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600d3890 con 0x7fb7600ba090 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb766159640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600d4090 con 0x7fb7600b73d0 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb75f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.435+0000 7fb75f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44626/0 (socket says 172.21.15.181:44626) 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 -- 172.21.15.181:0/4116464358 learned_addr learned my addr 172.21.15.181:0/4116464358 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75ffff640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7600ba090 0x7fb76015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75effd640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7600b73d0 0x7fb760160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 -- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7600b73d0 msgr2=0x7fb760160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7600b73d0 0x7fb760160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 -- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7600ba090 msgr2=0x7fb76015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7600ba090 0x7fb76015f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 -- 172.21.15.181:0/4116464358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760160170 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb75f7fe640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb75002b9e0 tx=0x7fb750004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750048070 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb750030660 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb760160450 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750034040 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.436+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb760158f20 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.437+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb760075890 con 0x7fb7600b5dc0 2024-03-29T04:10:00.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.438+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb75004b070 con 0x7fb7600b5dc0 2024-03-29T04:10:00.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.438+0000 7fb747fff640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 0x7fb74c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.438+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb7500ad900 con 0x7fb7600b5dc0 2024-03-29T04:10:00.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.438+0000 7fb75effd640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 0x7fb74c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.438+0000 7fb75effd640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 0x7fb74c06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb760159800 tx=0x7fb754009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.442+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb75007b560 con 0x7fb7600b5dc0 2024-03-29T04:10:00.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.617+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7600d2220 con 0x7fb7600b5dc0 2024-03-29T04:10:00.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.617+0000 7fb747fff640 1 -- 172.21.15.181:0/4116464358 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb75007f860 con 0x7fb7600b5dc0 2024-03-29T04:10:00.597 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 msgr2=0x7fb74c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 0x7fb74c06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb760159800 tx=0x7fb754009040 comp rx=0 tx=0).stop 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 msgr2=0x7fb7601589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb75002b9e0 tx=0x7fb750004730 comp rx=0 tx=0).stop 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 shutdown_connections 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7600ba090 0x7fb76015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb74c06c530 0x7fb74c06e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7600b73d0 0x7fb760160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 --2- 172.21.15.181:0/4116464358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7600b5dc0 0x7fb7601589e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 >> 172.21.15.181:0/4116464358 conn(0x7fb7600678f0 msgr2=0x7fb7600b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 shutdown_connections 2024-03-29T04:10:00.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.619+0000 7fb766159640 1 -- 172.21.15.181:0/4116464358 wait complete. 2024-03-29T04:10:00.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:00.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.733+0000 7fcc00c38640 1 Processor -- start 2024-03-29T04:10:00.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.733+0000 7fcc00c38640 1 -- start start 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcc00c38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcc00c38640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc0b4d70 con 0x7fcbfc0718c0 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcc00c38640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc0b35c0 con 0x7fcbfc15aaf0 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcc00c38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc0b41d0 con 0x7fcbfc15a710 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfad9d640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcbfc0718c0 0x7fcbfc06d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48660/0 (socket says 172.21.15.181:48660) 2024-03-29T04:10:00.714 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfad9d640 1 -- 172.21.15.181:0/424434110 learned_addr learned my addr 172.21.15.181:0/424434110 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfa59c640 1 --2- 172.21.15.181:0/424434110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292158893 0 0) 0x7fcbfc0b4d70 con 0x7fcbfc0718c0 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe4003520 con 0x7fcbfc0718c0 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312926169 0 0) 0x7fcbfc0b35c0 con 0x7fcbfc15aaf0 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbfc0b4d70 con 0x7fcbfc15aaf0 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfa59c640 1 -- 172.21.15.181:0/424434110 >> v1:172.21.15.187:6789/0 conn(0x7fcbfc15aaf0 legacy=0x7fcbfc0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfa59c640 1 -- 172.21.15.181:0/424434110 >> v1:172.21.15.181:6789/0 conn(0x7fcbfc0718c0 legacy=0x7fcbfc06d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.734+0000 7fcbfa59c640 1 -- 172.21.15.181:0/424434110 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0b38a0 con 0x7fcbfc15a710 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.735+0000 7fcbfa59c640 1 --2- 172.21.15.181:0/424434110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc0b2f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009dc0 tx=0x7fcbec02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ab66d7f92fae19 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.735+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec039070 con 0x7fcbfc15a710 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.735+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec02cc40 con 0x7fcbfc15a710 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.735+0000 7fcbf959a640 1 -- 172.21.15.181:0/424434110 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec034440 con 0x7fcbfc15a710 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 -- 172.21.15.181:0/424434110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 msgr2=0x7fcbfc0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 --2- 172.21.15.181:0/424434110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc0b2f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009dc0 tx=0x7fcbec02be90 comp rx=0 tx=0).stop 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 -- 172.21.15.181:0/424434110 shutdown_connections 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 --2- 172.21.15.181:0/424434110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc0b2f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 -- 172.21.15.181:0/424434110 >> 172.21.15.181:0/424434110 conn(0x7fcbfc0b5b00 msgr2=0x7fcbfc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 -- 172.21.15.181:0/424434110 shutdown_connections 2024-03-29T04:10:00.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.736+0000 7fcc00c38640 1 -- 172.21.15.181:0/424434110 wait complete. 2024-03-29T04:10:00.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.737+0000 7fcc00c38640 1 Processor -- start 2024-03-29T04:10:00.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.737+0000 7fcc00c38640 1 -- start start 2024-03-29T04:10:00.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc15aaf0 0x7fcbfc159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc072c50 con 0x7fcbfc0718c0 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0b3e60 con 0x7fcbfc15aaf0 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcc00c38640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc06d8f0 con 0x7fcbfc15a710 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44636/0 (socket says 172.21.15.181:44636) 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 -- 172.21.15.181:0/3611008474 learned_addr learned my addr 172.21.15.181:0/3611008474 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfad9d640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc15aaf0 0x7fcbfc159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbf9d9b640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 -- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 msgr2=0x7fcbfc1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 -- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc15aaf0 msgr2=0x7fcbfc159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc15aaf0 0x7fcbfc159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.738+0000 7fcbfa59c640 1 -- 172.21.15.181:0/3611008474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc154da0 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcbfa59c640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009dc0 tx=0x7fcbec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec044020 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec034d10 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc15c110 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec039040 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.739+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc151070 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.740+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc075890 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.740+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcbec047020 con 0x7fcbfc0718c0 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.740+0000 7fcbeaffd640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 0x7fcbdc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:00.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.741+0000 7fcbf9d9b640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 0x7fcbdc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:00.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.741+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcbec0ae140 con 0x7fcbfc0718c0 2024-03-29T04:10:00.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.741+0000 7fcbf9d9b640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 0x7fcbdc06e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:00.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.745+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcbec04e030 con 0x7fcbfc0718c0 2024-03-29T04:10:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.920+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcbfc06dde0 con 0x7fcbfc0718c0 2024-03-29T04:10:00.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.921+0000 7fcbeaffd640 1 -- 172.21.15.181:0/3611008474 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fcbec07bda0 con 0x7fcbfc0718c0 2024-03-29T04:10:00.900 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 msgr2=0x7fcbdc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 0x7fcbdc06e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 msgr2=0x7fcbfc150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009dc0 tx=0x7fcbec004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 shutdown_connections 2024-03-29T04:10:00.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbfc15aaf0 0x7fcbfc159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbdc06c530 0x7fcbdc06e9f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbfc15a710 0x7fcbfc1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 --2- 172.21.15.181:0/3611008474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbfc0718c0 0x7fcbfc150b30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 >> 172.21.15.181:0/3611008474 conn(0x7fcbfc0b5b00 msgr2=0x7fcbfc152960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 shutdown_connections 2024-03-29T04:10:00.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:00.923+0000 7fcc00c38640 1 -- 172.21.15.181:0/3611008474 wait complete. 2024-03-29T04:10:00.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:01.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.038+0000 7fc80d39f640 1 Processor -- start 2024-03-29T04:10:01.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.038+0000 7fc80d39f640 1 -- start start 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80d39f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4d00 0x7fc8080d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80d39f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8080d0b90 0x7fc8080d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80d39f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8080b54b0 con 0x7fc8080b4920 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80d39f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc8080d0260 con 0x7fc8080d0b90 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80d39f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8080cfca0 con 0x7fc8080b4d00 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8080d0b90 0x7fc8080d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc807fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc8080b4920 0x7fc8080b3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48666/0 (socket says 172.21.15.181:48666) 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8080d0b90 0x7fc8080d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44640/0 (socket says 172.21.15.181:44640) 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc807fff640 1 -- 172.21.15.181:0/1619956328 learned_addr learned my addr 172.21.15.181:0/1619956328 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc8077fe640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4d00 0x7fc8080d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.019 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 -- 172.21.15.181:0/1619956328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4d00 msgr2=0x7fc8080d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:01.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4d00 0x7fc8080d4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 -- 172.21.15.181:0/1619956328 >> v1:172.21.15.181:6789/0 conn(0x7fc8080b4920 legacy=0x7fc8080b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc80cb9e640 1 -- 172.21.15.181:0/1619956328 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080cff80 con 0x7fc8080d0b90 2024-03-29T04:10:01.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.039+0000 7fc806ffd640 1 -- 172.21.15.181:0/1619956328 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047000495 0 0) 0x7fc7f8001110 con 0x7fc8080b4920 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.040+0000 7fc80cb9e640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080d0b90 0x7fc8080d1040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d90 tx=0x7fc80002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435f29aeabd7a3a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.040+0000 7fc806ffd640 1 -- 172.21.15.181:0/1619956328 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800039070 con 0x7fc8080d0b90 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.040+0000 7fc806ffd640 1 -- 172.21.15.181:0/1619956328 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80002cbc0 con 0x7fc8080d0b90 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.040+0000 7fc806ffd640 1 -- 172.21.15.181:0/1619956328 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800034440 con 0x7fc8080d0b90 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 -- 172.21.15.181:0/1619956328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080d0b90 msgr2=0x7fc8080d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080d0b90 0x7fc8080d1040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d90 tx=0x7fc80002be10 comp rx=0 tx=0).stop 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 -- 172.21.15.181:0/1619956328 shutdown_connections 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080d0b90 0x7fc8080d1040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 --2- 172.21.15.181:0/1619956328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4d00 0x7fc8080d4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 -- 172.21.15.181:0/1619956328 >> 172.21.15.181:0/1619956328 conn(0x7fc80809d4e0 msgr2=0x7fc8080d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 -- 172.21.15.181:0/1619956328 shutdown_connections 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.041+0000 7fc80d39f640 1 -- 172.21.15.181:0/1619956328 wait complete. 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.042+0000 7fc80d39f640 1 Processor -- start 2024-03-29T04:10:01.022 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.042+0000 7fc80d39f640 1 -- start start 2024-03-29T04:10:01.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4920 0x7fc808155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 0x7fc8081f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc8080d5000 con 0x7fc8080b4d00 2024-03-29T04:10:01.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8080d2180 con 0x7fc8080d0b90 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80d39f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8080cf8c0 con 0x7fc8080b4920 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44648/0 (socket says 172.21.15.181:44648) 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 -- 172.21.15.181:0/3493292310 learned_addr learned my addr 172.21.15.181:0/3493292310 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc807fff640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4920 0x7fc808155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80cb9e640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 0x7fc8081f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 -- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4920 msgr2=0x7fc808155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4920 0x7fc808155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 -- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 msgr2=0x7fc8081f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 0x7fc8081f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 -- 172.21.15.181:0/3493292310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080c8f50 con 0x7fc8080b4d00 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc80cb9e640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 0x7fc8081f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc8077fe640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e9b0 tx=0x7fc7fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.043+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc017070 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.044+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00c9a0 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.044+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8080c9230 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.044+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cd90 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.044+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8080c97f0 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.045+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8080cd670 con 0x7fc8080b4d00 2024-03-29T04:10:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.045+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc7fc01a070 con 0x7fc8080b4d00 2024-03-29T04:10:01.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.046+0000 7fc804ff9640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 0x7fc7f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.046+0000 7fc807fff640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 0x7fc7f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.046+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc7fc08ba10 con 0x7fc8080b4d00 2024-03-29T04:10:01.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.046+0000 7fc807fff640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 0x7fc7f406e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8007fd0 tx=0x7fc7f8007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.050+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc8080cd670 con 0x7fc8080b4d00 2024-03-29T04:10:01.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.224+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc808075890 con 0x7fc8080b4d00 2024-03-29T04:10:01.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.224+0000 7fc804ff9640 1 -- 172.21.15.181:0/3493292310 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc7fc0595e0 con 0x7fc8080b4d00 2024-03-29T04:10:01.204 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:10:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 msgr2=0x7fc7f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 0x7fc7f406e9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8007fd0 tx=0x7fc7f8007de0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 msgr2=0x7fc8080c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e9b0 tx=0x7fc7fc00ee70 comp rx=0 tx=0).stop 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 shutdown_connections 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8080d0b90 0x7fc8081f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8080b4d00 0x7fc8080c9ff0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc7f406c530 0x7fc7f406e9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 --2- 172.21.15.181:0/3493292310 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8080b4920 0x7fc808155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.226+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 >> 172.21.15.181:0/3493292310 conn(0x7fc80809d4e0 msgr2=0x7fc80806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.227+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 shutdown_connections 2024-03-29T04:10:01.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.227+0000 7fc80d39f640 1 -- 172.21.15.181:0/3493292310 wait complete. 2024-03-29T04:10:01.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.340+0000 7f423b2ba640 1 Processor -- start 2024-03-29T04:10:01.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.340+0000 7f423b2ba640 1 -- start start 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f423b2ba640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4234144cb0 0x7f42340b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f423b2ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234072d50 0x7f42340ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f423b2ba640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42340b35e0 con 0x7f423415f800 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f423b2ba640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f42340b3e80 con 0x7f4234144cb0 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f423b2ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42340b38c0 con 0x7f4234072d50 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239056640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4234144cb0 0x7f42340b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4238855640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f423415f800 0x7f4234072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48676/0 (socket says 172.21.15.181:48676) 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4238855640 1 -- 172.21.15.181:0/2571317715 learned_addr learned my addr 172.21.15.181:0/2571317715 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:01.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239857640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234072d50 0x7f42340ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239056640 1 -- 172.21.15.181:0/2571317715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234072d50 msgr2=0x7f42340ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239056640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234072d50 0x7f42340ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239056640 1 -- 172.21.15.181:0/2571317715 >> v1:172.21.15.181:6789/0 conn(0x7f423415f800 legacy=0x7f4234072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f422bfff640 1 -- 172.21.15.181:0/2571317715 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281504830 0 0) 0x7f42340b35e0 con 0x7f423415f800 2024-03-29T04:10:01.322 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.341+0000 7f4239056640 1 -- 172.21.15.181:0/2571317715 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42340b3ba0 con 0x7f4234144cb0 2024-03-29T04:10:01.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.342+0000 7f4239056640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4234144cb0 0x7f42340b2f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42300097a0 tx=0x7f423002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f63225c2d705c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.342+0000 7f422bfff640 1 -- 172.21.15.181:0/2571317715 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230035020 con 0x7f4234144cb0 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.342+0000 7f422bfff640 1 -- 172.21.15.181:0/2571317715 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4230031830 con 0x7f4234144cb0 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.342+0000 7f422bfff640 1 -- 172.21.15.181:0/2571317715 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230031c00 con 0x7f4234144cb0 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 -- 172.21.15.181:0/2571317715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4234144cb0 msgr2=0x7f42340b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4234144cb0 0x7f42340b2f20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42300097a0 tx=0x7f423002b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 -- 172.21.15.181:0/2571317715 shutdown_connections 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234072d50 0x7f42340ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 --2- 172.21.15.181:0/2571317715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4234144cb0 0x7f42340b2f20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 -- 172.21.15.181:0/2571317715 >> 172.21.15.181:0/2571317715 conn(0x7f42340b5b00 msgr2=0x7f423406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 -- 172.21.15.181:0/2571317715 shutdown_connections 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.343+0000 7f423b2ba640 1 -- 172.21.15.181:0/2571317715 wait complete. 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.344+0000 7f423b2ba640 1 Processor -- start 2024-03-29T04:10:01.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.344+0000 7f423b2ba640 1 -- start start 2024-03-29T04:10:01.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234072d50 0x7f4234158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234144cb0 0x7f4234167820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42340ce590 con 0x7f423415f800 2024-03-29T04:10:01.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f423415fc00 con 0x7f4234072d50 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42340ce710 con 0x7f4234144cb0 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44674/0 (socket says 172.21.15.181:44674) 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 -- 172.21.15.181:0/422438125 learned_addr learned my addr 172.21.15.181:0/422438125 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4238855640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234144cb0 0x7f4234167820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239056640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234072d50 0x7f4234158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 -- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234144cb0 msgr2=0x7f4234167820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234144cb0 0x7f4234167820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 -- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234072d50 msgr2=0x7f4234158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234072d50 0x7f4234158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 -- 172.21.15.181:0/422438125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4234158950 con 0x7f423415f800 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4239857640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f422c00ef90 tx=0x7f422c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c018020 con 0x7f423415f800 2024-03-29T04:10:01.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422c014480 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4234158c30 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.345+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c014870 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.346+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42341591f0 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.347+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4234075890 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.347+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f422c005d10 con 0x7f423415f800 2024-03-29T04:10:01.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.347+0000 7f4229ffb640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.347+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f422c08c7c0 con 0x7f423415f800 2024-03-29T04:10:01.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.347+0000 7f4239056640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.348+0000 7f4239056640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 0x7f421806ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4230036000 tx=0x7f4230008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.352+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f422c022030 con 0x7f423415f800 2024-03-29T04:10:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.526+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f42340b4030 con 0x7f423415f800 2024-03-29T04:10:01.506 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.527+0000 7f4229ffb640 1 -- 172.21.15.181:0/422438125 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f422c010040 con 0x7f423415f800 2024-03-29T04:10:01.507 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:01.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 msgr2=0x7f421806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 0x7f421806ea20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4230036000 tx=0x7f4230008030 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 msgr2=0x7f4234167d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f422c00ef90 tx=0x7f422c00c490 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 shutdown_connections 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423415f800 0x7f4234167d60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4234144cb0 0x7f4234167820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 --2- 172.21.15.181:0/422438125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4234072d50 0x7f4234158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 >> 172.21.15.181:0/422438125 conn(0x7f42340b5b00 msgr2=0x7f42341438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 shutdown_connections 2024-03-29T04:10:01.509 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.529+0000 7f423b2ba640 1 -- 172.21.15.181:0/422438125 wait complete. 2024-03-29T04:10:01.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:01.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.641+0000 7f88eeee4640 1 Processor -- start 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.641+0000 7f88eeee4640 1 -- start start 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88eeee4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88e8071440 0x7f88e80cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88eeee4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e80cca60 0x7f88e80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88eeee4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e8071fe0 con 0x7f88e815fba0 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88eeee4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f88e80cd3f0 con 0x7f88e80cca60 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88eeee4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f88e8071a20 con 0x7f88e8071440 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88ed481640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e80cca60 0x7f88e80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88ecc80640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f88e815fba0 0x7f88e8160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48686/0 (socket says 172.21.15.181:48686) 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88ed481640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88e80cca60 0x7f88e80cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44684/0 (socket says 172.21.15.181:44684) 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88ecc80640 1 -- 172.21.15.181:0/3771692953 learned_addr learned my addr 172.21.15.181:0/3771692953 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.642+0000 7f88e7fff640 1 --2- 172.21.15.181:0/3771692953 >> v2:172.21.15.187:3300/0 conn(0x7f88e8071440 0x7f88e80cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88ed481640 1 -- 172.21.15.181:0/3771692953 >> v2:172.21.15.187:3300/0 conn(0x7f88e8071440 msgr2=0x7f88e80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:01.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88ed481640 1 --2- 172.21.15.181:0/3771692953 >> v2:172.21.15.187:3300/0 conn(0x7f88e8071440 0x7f88e80cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88ed481640 1 -- 172.21.15.181:0/3771692953 >> v1:172.21.15.181:6789/0 conn(0x7f88e815fba0 legacy=0x7f88e8160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88ed481640 1 -- 172.21.15.181:0/3771692953 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8071d00 con 0x7f88e80cca60 2024-03-29T04:10:01.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88e77fe640 1 -- 172.21.15.181:0/3771692953 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234138210 0 0) 0x7f88e8071fe0 con 0x7f88e815fba0 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88ed481640 1 --2- 172.21.15.181:0/3771692953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80cceb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f88e80d0fa0 tx=0x7f88e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4718b7e93342b3e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88e77fe640 1 -- 172.21.15.181:0/3771692953 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0039070 con 0x7f88e80cca60 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88e77fe640 1 -- 172.21.15.181:0/3771692953 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e002cc80 con 0x7f88e80cca60 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.643+0000 7f88e77fe640 1 -- 172.21.15.181:0/3771692953 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0034660 con 0x7f88e80cca60 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 -- 172.21.15.181:0/3771692953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 msgr2=0x7f88e80cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3771692953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80cceb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f88e80d0fa0 tx=0x7f88e002bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 -- 172.21.15.181:0/3771692953 shutdown_connections 2024-03-29T04:10:01.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3771692953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80cceb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3771692953 >> v2:172.21.15.187:3300/0 conn(0x7f88e8071440 0x7f88e80cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 -- 172.21.15.181:0/3771692953 >> 172.21.15.181:0/3771692953 conn(0x7f88e806d610 msgr2=0x7f88e806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 -- 172.21.15.181:0/3771692953 shutdown_connections 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.644+0000 7f88eeee4640 1 -- 172.21.15.181:0/3771692953 wait complete. 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.645+0000 7f88eeee4640 1 Processor -- start 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.645+0000 7f88eeee4640 1 -- start start 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88e8071440 0x7f88e80b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88e815fba0 0x7f88e81f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88e80cb930 con 0x7f88e80cca60 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88e80caed0 con 0x7f88e815fba0 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88eeee4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88e80b2f20 con 0x7f88e8071440 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44690/0 (socket says 172.21.15.181:44690) 2024-03-29T04:10:01.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 -- 172.21.15.181:0/3662501903 learned_addr learned my addr 172.21.15.181:0/3662501903 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88ecc80640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88e8071440 0x7f88e80b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88ed481640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88e815fba0 0x7f88e81f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 -- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88e8071440 msgr2=0x7f88e80b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88e8071440 0x7f88e80b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 -- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88e815fba0 msgr2=0x7f88e81f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88e815fba0 0x7f88e81f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.646+0000 7f88e7fff640 1 -- 172.21.15.181:0/3662501903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e80cdf60 con 0x7f88e80cca60 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88e7fff640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00e9e0 tx=0x7f88dc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc017070 con 0x7f88e80cca60 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc00c9d0 con 0x7f88e80cca60 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00cdc0 con 0x7f88e80cca60 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81f5d60 con 0x7f88e80cca60 2024-03-29T04:10:01.629 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.647+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e80c8590 con 0x7f88e80cca60 2024-03-29T04:10:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.648+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8075900 con 0x7f88e80cca60 2024-03-29T04:10:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.648+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f88dc01a070 con 0x7f88e80cca60 2024-03-29T04:10:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.649+0000 7f88e57fa640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 0x7f88d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.649+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f88dc08ba40 con 0x7f88e80cca60 2024-03-29T04:10:01.630 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.649+0000 7f88ecc80640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 0x7f88d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:01.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.649+0000 7f88ecc80640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 0x7f88d406e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88d8007fd0 tx=0x7f88d8007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:01.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.653+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f88dc059610 con 0x7f88e80cca60 2024-03-29T04:10:01.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.838+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88e8073050 con 0x7f88e80cca60 2024-03-29T04:10:01.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.838+0000 7f88e57fa640 1 -- 172.21.15.181:0/3662501903 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f88dc05d910 con 0x7f88e80cca60 2024-03-29T04:10:01.818 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 msgr2=0x7f88d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 0x7f88d406e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88d8007fd0 tx=0x7f88d8007de0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 msgr2=0x7f88e80b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00e9e0 tx=0x7f88dc00eea0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 shutdown_connections 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88e815fba0 0x7f88e81f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88e80cca60 0x7f88e80b1210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f88d406c530 0x7f88d406e9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 --2- 172.21.15.181:0/3662501903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88e8071440 0x7f88e80b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:01.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 >> 172.21.15.181:0/3662501903 conn(0x7f88e806d610 msgr2=0x7f88e80b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:01.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 shutdown_connections 2024-03-29T04:10:01.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:01.840+0000 7f88eeee4640 1 -- 172.21.15.181:0/3662501903 wait complete. 2024-03-29T04:10:01.831 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-29T04:10:01.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:01.831 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-29T04:10:01.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:10:01.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:01.860 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:10:01.914 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:01.914 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:01.930 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:01.930 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:01.985 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:01.986 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:02.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:02.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.163+0000 7f8b22159640 1 Processor -- start 2024-03-29T04:10:02.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.163+0000 7f8b22159640 1 -- start start 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8b1c0d48c0 0x7f8b1c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c0d3cb0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1c0d4d00 con 0x7f8b1c0d48c0 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b22159640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c0d36f0 con 0x7f8b1c15e950 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44692/0 (socket says 172.21.15.181:44692) 2024-03-29T04:10:02.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 -- 172.21.15.181:0/1439670701 learned_addr learned my addr 172.21.15.181:0/1439670701 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1affd640 1 --2- 172.21.15.181:0/1439670701 >> v2:172.21.15.187:3300/0 conn(0x7f8b1c0d48c0 0x7f8b1c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1b7fe640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 -- 172.21.15.181:0/1439670701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 msgr2=0x7f8b1c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 -- 172.21.15.181:0/1439670701 >> v2:172.21.15.187:3300/0 conn(0x7f8b1c0d48c0 msgr2=0x7f8b1c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 --2- 172.21.15.181:0/1439670701 >> v2:172.21.15.187:3300/0 conn(0x7f8b1c0d48c0 0x7f8b1c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.164+0000 7f8b1bfff640 1 -- 172.21.15.181:0/1439670701 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c0d39d0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.165+0000 7f8b1bfff640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0736a0 tx=0x7f8b1402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdaad1333c8c96d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.165+0000 7f8b1a7fc640 1 -- 172.21.15.181:0/1439670701 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1402cb00 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.165+0000 7f8b1a7fc640 1 -- 172.21.15.181:0/1439670701 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1402cca0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.165+0000 7f8b1a7fc640 1 -- 172.21.15.181:0/1439670701 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14034610 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 -- 172.21.15.181:0/1439670701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 msgr2=0x7f8b1c0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0736a0 tx=0x7f8b1402bf60 comp rx=0 tx=0).stop 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 -- 172.21.15.181:0/1439670701 shutdown_connections 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c0cef90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 --2- 172.21.15.181:0/1439670701 >> v2:172.21.15.187:3300/0 conn(0x7f8b1c0d48c0 0x7f8b1c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 --2- 172.21.15.181:0/1439670701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 -- 172.21.15.181:0/1439670701 >> 172.21.15.181:0/1439670701 conn(0x7f8b1c0676e0 msgr2=0x7f8b1c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 -- 172.21.15.181:0/1439670701 shutdown_connections 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.166+0000 7f8b22159640 1 -- 172.21.15.181:0/1439670701 wait complete. 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.167+0000 7f8b22159640 1 Processor -- start 2024-03-29T04:10:02.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.167+0000 7f8b22159640 1 -- start start 2024-03-29T04:10:02.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0d48c0 0x7f8b1c1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0d3390 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c0b7da0 con 0x7f8b1c0d48c0 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b22159640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1c15ed30 con 0x7f8b1c15e950 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44708/0 (socket says 172.21.15.181:44708) 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 -- 172.21.15.181:0/192149576 learned_addr learned my addr 172.21.15.181:0/192149576 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1bfff640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1affd640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0d48c0 0x7f8b1c1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 -- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 msgr2=0x7f8b1c0aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 -- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0d48c0 msgr2=0x7f8b1c1f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0d48c0 0x7f8b1c1f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 -- 172.21.15.181:0/192149576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c1617c0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.168+0000 7f8b1b7fe640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00e9b0 tx=0x7f8b0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.169+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c017070 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.169+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0c00c9a0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.169+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c160490 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.169+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00cd90 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.169+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c160a50 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.170+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c075a10 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.170+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b0c01a070 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.170+0000 7f8b03fff640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 0x7f8b0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.170+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8b0c08b9e0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.170+0000 7f8b1affd640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 0x7f8b0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.171+0000 7f8b1affd640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 0x7f8b0806e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c15aed0 tx=0x7f8b10009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.175+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8b0c0595b0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.351+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b1c067e80 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.352+0000 7f8b03fff640 1 -- 172.21.15.181:0/192149576 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7f8b0c05d8b0 con 0x7f8b1c0d2f00 2024-03-29T04:10:02.332 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:02.332 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 msgr2=0x7f8b0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 0x7f8b0806e9f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c15aed0 tx=0x7f8b10009190 comp rx=0 tx=0).stop 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 msgr2=0x7f8b1c15a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00e9b0 tx=0x7f8b0c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 shutdown_connections 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8b0806c530 0x7f8b0806e9f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8b1c15e950 0x7f8b1c0aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1c0d48c0 0x7f8b1c1f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 --2- 172.21.15.181:0/192149576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b1c0d2f00 0x7f8b1c15a4c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 >> 172.21.15.181:0/192149576 conn(0x7f8b1c0676e0 msgr2=0x7f8b1c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 shutdown_connections 2024-03-29T04:10:02.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.354+0000 7f8b22159640 1 -- 172.21.15.181:0/192149576 wait complete. 2024-03-29T04:10:02.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:02.345 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:02.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.470+0000 7fbb40006640 1 Processor -- start 2024-03-29T04:10:02.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.470+0000 7fbb40006640 1 -- start start 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb40006640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380ba090 0x7fbb380d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb40006640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb380b73d0 0x7fbb380b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb40006640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb380d2ff0 con 0x7fbb380b5dc0 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb40006640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fbb380d32d0 con 0x7fbb380b73d0 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb40006640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380d35b0 con 0x7fbb380ba090 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb3e5a3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb380b73d0 0x7fbb380b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb3e5a3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb380b73d0 0x7fbb380b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44722/0 (socket says 172.21.15.181:44722) 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/2935673204 learned_addr learned my addr 172.21.15.181:0/2935673204 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:02.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.471+0000 7fbb3d5a1640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380ba090 0x7fbb380d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.472+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/2935673204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380ba090 msgr2=0x7fbb380d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.472+0000 7fbb3e5a3640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380ba090 0x7fbb380d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.472+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/2935673204 >> v1:172.21.15.187:6789/0 conn(0x7fbb380b5dc0 legacy=0x7fbb380b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.472+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/2935673204 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380d3890 con 0x7fbb380b73d0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.472+0000 7fbb3e5a3640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380b73d0 0x7fbb380b7880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009870 tx=0x7fbb3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ea9034a79a2e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb3cda0640 1 -- 172.21.15.181:0/2935673204 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30034070 con 0x7fbb380b73d0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb3cda0640 1 -- 172.21.15.181:0/2935673204 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb30030840 con 0x7fbb380b73d0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb3cda0640 1 -- 172.21.15.181:0/2935673204 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30030c10 con 0x7fbb380b73d0 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 -- 172.21.15.181:0/2935673204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380b73d0 msgr2=0x7fbb380b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380b73d0 0x7fbb380b7880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009870 tx=0x7fbb3002b980 comp rx=0 tx=0).stop 2024-03-29T04:10:02.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 -- 172.21.15.181:0/2935673204 shutdown_connections 2024-03-29T04:10:02.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380b73d0 0x7fbb380b7880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 --2- 172.21.15.181:0/2935673204 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380ba090 0x7fbb380d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 -- 172.21.15.181:0/2935673204 >> 172.21.15.181:0/2935673204 conn(0x7fbb380678f0 msgr2=0x7fbb380d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 -- 172.21.15.181:0/2935673204 shutdown_connections 2024-03-29T04:10:02.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.473+0000 7fbb40006640 1 -- 172.21.15.181:0/2935673204 wait complete. 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.474+0000 7fbb40006640 1 Processor -- start 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.474+0000 7fbb40006640 1 -- start start 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb380b5dc0 0x7fbb380ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380b73d0 0x7fbb380b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3815e880 con 0x7fbb380ba090 2024-03-29T04:10:02.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380d4090 con 0x7fbb380b5dc0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb40006640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3815ea00 con 0x7fbb380b73d0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3e5a3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3e5a3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44738/0 (socket says 172.21.15.181:44738) 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/4034065762 learned_addr learned my addr 172.21.15.181:0/4034065762 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3dda2640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb380b5dc0 0x7fbb380ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3d5a1640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380b73d0 0x7fbb380b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380b73d0 msgr2=0x7fbb380b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.475+0000 7fbb3e5a3640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380b73d0 0x7fbb380b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb380b5dc0 msgr2=0x7fbb380ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb3e5a3640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb380b5dc0 0x7fbb380ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb3e5a3640 1 -- 172.21.15.181:0/4034065762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3815edd0 con 0x7fbb380ba090 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb3e5a3640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002b9c0 tx=0x7fbb30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30048070 con 0x7fbb380ba090 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb30030640 con 0x7fbb380ba090 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3815f0b0 con 0x7fbb380ba090 2024-03-29T04:10:02.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30034040 con 0x7fbb380ba090 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.476+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3815fb90 con 0x7fbb380ba090 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.477+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3815fe20 con 0x7fbb380ba090 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.477+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbb30037070 con 0x7fbb380ba090 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.478+0000 7fbb2e7fc640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 0x7fbb1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.478+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbb300ad700 con 0x7fbb380ba090 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.478+0000 7fbb3dda2640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 0x7fbb1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.478+0000 7fbb3dda2640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 0x7fbb1c06e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb34001f10 tx=0x7fbb34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.482+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbb3007b370 con 0x7fbb380ba090 2024-03-29T04:10:02.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.655+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb38075890 con 0x7fbb380ba090 2024-03-29T04:10:02.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.655+0000 7fbb2e7fc640 1 -- 172.21.15.181:0/4034065762 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fbb3007f670 con 0x7fbb380ba090 2024-03-29T04:10:02.635 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 msgr2=0x7fbb1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 0x7fbb1c06e9f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb34001f10 tx=0x7fbb34009040 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 msgr2=0x7fbb380b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002b9c0 tx=0x7fbb30004730 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 shutdown_connections 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb380ba090 0x7fbb380b5630 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbb1c06c530 0x7fbb1c06e9f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb380b73d0 0x7fbb380b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 --2- 172.21.15.181:0/4034065762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb380b5dc0 0x7fbb380ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 >> 172.21.15.181:0/4034065762 conn(0x7fbb380678f0 msgr2=0x7fbb380b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 shutdown_connections 2024-03-29T04:10:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.657+0000 7fbb40006640 1 -- 172.21.15.181:0/4034065762 wait complete. 2024-03-29T04:10:02.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:02.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.772+0000 7f042acfb640 1 Processor -- start 2024-03-29T04:10:02.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.772+0000 7f042acfb640 1 -- start start 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.772+0000 7f042acfb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f042415e5b0 0x7f0424162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f042acfb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f042acfb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04241630b0 con 0x7f04240d1be0 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f042acfb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f0424073880 con 0x7f042415e5b0 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f042acfb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04240d0e20 con 0x7f0424073310 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0428a97640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f042415e5b0 0x7f0424162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0429298640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f04240d1be0 0x7f04240b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48688/0 (socket says 172.21.15.181:48688) 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0429298640 1 -- 172.21.15.181:0/1161075910 learned_addr learned my addr 172.21.15.181:0/1161075910 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0423fff640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0428a97640 1 -- 172.21.15.181:0/1161075910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 msgr2=0x7f042406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0428a97640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0428a97640 1 -- 172.21.15.181:0/1161075910 >> v1:172.21.15.181:6789/0 conn(0x7f04240d1be0 legacy=0x7f04240b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f0428a97640 1 -- 172.21.15.181:0/1161075910 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04240d1100 con 0x7f042415e5b0 2024-03-29T04:10:02.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.773+0000 7f04237fe640 1 -- 172.21.15.181:0/1161075910 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935649805 0 0) 0x7f04241630b0 con 0x7f04240d1be0 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f0428a97640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f0424162b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8843e1a42a11e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f04237fe640 1 -- 172.21.15.181:0/1161075910 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04180315a0 con 0x7f042415e5b0 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f04237fe640 1 -- 172.21.15.181:0/1161075910 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418037070 con 0x7f042415e5b0 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f04237fe640 1 -- 172.21.15.181:0/1161075910 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418031960 con 0x7f042415e5b0 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 -- 172.21.15.181:0/1161075910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 msgr2=0x7f0424162b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f0424162b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04180097a0 tx=0x7f041802b840 comp rx=0 tx=0).stop 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 -- 172.21.15.181:0/1161075910 shutdown_connections 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 --2- 172.21.15.181:0/1161075910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f0424162b70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 -- 172.21.15.181:0/1161075910 >> 172.21.15.181:0/1161075910 conn(0x7f042406e720 msgr2=0x7f042406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 -- 172.21.15.181:0/1161075910 shutdown_connections 2024-03-29T04:10:02.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.774+0000 7f042acfb640 1 -- 172.21.15.181:0/1161075910 wait complete. 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 Processor -- start 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 -- start start 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04240d1be0 0x7f04240ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f042415e990 con 0x7f042415e5b0 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04240d16c0 con 0x7f04240d1be0 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.776+0000 7f042acfb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f042406bac0 con 0x7f0424073310 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44750/0 (socket says 172.21.15.181:44750) 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 -- 172.21.15.181:0/1196643940 learned_addr learned my addr 172.21.15.181:0/1196643940 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0428a97640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0423fff640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04240d1be0 0x7f04240ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 -- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 msgr2=0x7f042415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 -- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04240d1be0 msgr2=0x7f04240ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04240d1be0 0x7f04240ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 -- 172.21.15.181:0/1196643940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424150880 con 0x7f042415e5b0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f0429298640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c7d0 tx=0x7f041c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c018070 con 0x7f042415e5b0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c00a640 con 0x7f042415e5b0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04240cd690 con 0x7f042415e5b0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00aa70 con 0x7f042415e5b0 2024-03-29T04:10:02.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.777+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04240cd930 con 0x7f042415e5b0 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.778+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0424075890 con 0x7f042415e5b0 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.779+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f041c01b070 con 0x7f042415e5b0 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.779+0000 7f04217fa640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 0x7f040806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.779+0000 7f0428a97640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 0x7f040806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.779+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f041c08b020 con 0x7f042415e5b0 2024-03-29T04:10:02.760 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.780+0000 7f0428a97640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 0x7f040806e9f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f041803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:02.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.783+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0424150880 con 0x7f042415e5b0 2024-03-29T04:10:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.958+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0424072be0 con 0x7f042415e5b0 2024-03-29T04:10:02.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.959+0000 7f04217fa640 1 -- 172.21.15.181:0/1196643940 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f041c058bf0 con 0x7f042415e5b0 2024-03-29T04:10:02.938 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 msgr2=0x7f040806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 0x7f040806e9f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0418036000 tx=0x7f041803d040 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 msgr2=0x7f042414fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c7d0 tx=0x7f041c00cc90 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 shutdown_connections 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042415e5b0 0x7f042414fa10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04240d1be0 0x7f04240ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f040806c530 0x7f040806e9f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 --2- 172.21.15.181:0/1196643940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0424073310 0x7f042415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 >> 172.21.15.181:0/1196643940 conn(0x7f042406e720 msgr2=0x7f0424159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 shutdown_connections 2024-03-29T04:10:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:02.961+0000 7f042acfb640 1 -- 172.21.15.181:0/1196643940 wait complete. 2024-03-29T04:10:02.951 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:03.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.075+0000 7fa6a75a8640 1 Processor -- start 2024-03-29T04:10:03.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.075+0000 7fa6a75a8640 1 -- start start 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a75a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a75a8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6a00d1be0 0x7fa6a006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a75a8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a0073880 con 0x7fa6a015e5b0 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a75a8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a00d0e20 con 0x7fa6a00d1be0 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a75a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a00d1100 con 0x7fa6a0073310 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a5b45640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa6a00d1be0 0x7fa6a006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a5344640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fa6a015e5b0 0x7fa6a0162b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:49626/0 (socket says 172.21.15.181:49626) 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a4b43640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.076+0000 7fa6a5344640 1 -- 172.21.15.181:0/2336471975 learned_addr learned my addr 172.21.15.181:0/2336471975 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:03.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa6a5b45640 1 -- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 msgr2=0x7fa6a006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa6a5b45640 1 --2- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa6a5b45640 1 -- 172.21.15.181:0/2336471975 >> v1:172.21.15.187:6789/0 conn(0x7fa6a015e5b0 legacy=0x7fa6a0162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa697fff640 1 -- 172.21.15.181:0/2336471975 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747518998 0 0) 0x7fa6a0073880 con 0x7fa6a015e5b0 2024-03-29T04:10:03.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa6a5b45640 1 -- 172.21.15.181:0/2336471975 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00d13e0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.077+0000 7fa6a5b45640 1 --2- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a00d1be0 0x7fa6a006bac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa6980097a0 tx=0x7fa69802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6063b052e23a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa697fff640 1 -- 172.21.15.181:0/2336471975 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6980315a0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa697fff640 1 -- 172.21.15.181:0/2336471975 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa698037070 con 0x7fa6a00d1be0 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa697fff640 1 -- 172.21.15.181:0/2336471975 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698031960 con 0x7fa6a00d1be0 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 -- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a00d1be0 msgr2=0x7fa6a006bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a00d1be0 0x7fa6a006bac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa6980097a0 tx=0x7fa69802b840 comp rx=0 tx=0).stop 2024-03-29T04:10:03.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 -- 172.21.15.181:0/2336471975 shutdown_connections 2024-03-29T04:10:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a00d1be0 0x7fa6a006bac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/2336471975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 -- 172.21.15.181:0/2336471975 >> 172.21.15.181:0/2336471975 conn(0x7fa6a006e720 msgr2=0x7fa6a006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 -- 172.21.15.181:0/2336471975 shutdown_connections 2024-03-29T04:10:03.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.078+0000 7fa6a75a8640 1 -- 172.21.15.181:0/2336471975 wait complete. 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.079+0000 7fa6a75a8640 1 Processor -- start 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.079+0000 7fa6a75a8640 1 -- start start 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a01437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a015e5b0 0x7fa6a006b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a006f9a0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a015e990 con 0x7fa6a015e5b0 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a75a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a006fb20 con 0x7fa6a0073310 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a5344640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a01437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a4b43640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a4b43640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44758/0 (socket says 172.21.15.181:44758) 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a4b43640 1 -- 172.21.15.181:0/1913549911 learned_addr learned my addr 172.21.15.181:0/1913549911 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a5344640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a01437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:41472/0 (socket says 172.21.15.181:41472) 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.080+0000 7fa6a5b45640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a015e5b0 0x7fa6a006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 -- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 msgr2=0x7fa6a01437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a01437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 -- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a015e5b0 msgr2=0x7fa6a006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a015e5b0 0x7fa6a006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 -- 172.21.15.181:0/1913549911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a0143d20 con 0x7fa6a00d1be0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a4b43640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e9e0 tx=0x7fa69000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa6a00d1be0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a0146310 con 0x7fa6a00d1be0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69000c9d0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000cdc0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.081+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a01468d0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.082+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a0075890 con 0x7fa6a00d1be0 2024-03-29T04:10:03.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.083+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa69001a070 con 0x7fa6a00d1be0 2024-03-29T04:10:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.083+0000 7fa695ffb640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 0x7fa68406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.083+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa69008ba90 con 0x7fa6a00d1be0 2024-03-29T04:10:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.083+0000 7fa6a5344640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 0x7fa68406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.084+0000 7fa6a5344640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 0x7fa68406ea20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0012e0 tx=0x7fa69c001630 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.087+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa6900595e0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.242 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.263+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6a0072be0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.243 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.264+0000 7fa695ffb640 1 -- 172.21.15.181:0/1913549911 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa69005d8e0 con 0x7fa6a00d1be0 2024-03-29T04:10:03.243 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:10:03.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 msgr2=0x7fa68406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.245 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 0x7fa68406ea20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0012e0 tx=0x7fa69c001630 comp rx=0 tx=0).stop 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 msgr2=0x7fa6a015f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e9e0 tx=0x7fa69000eea0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 shutdown_connections 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6a015e5b0 0x7fa6a006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a00d1be0 0x7fa6a015f090 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.246 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa68406c560 0x7fa68406ea20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 --2- 172.21.15.181:0/1913549911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa6a0073310 0x7fa6a01437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 >> 172.21.15.181:0/1913549911 conn(0x7fa6a006e720 msgr2=0x7fa6a00af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 shutdown_connections 2024-03-29T04:10:03.249 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.266+0000 7fa6a75a8640 1 -- 172.21.15.181:0/1913549911 wait complete. 2024-03-29T04:10:03.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:03.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.381+0000 7f67c1b9d640 1 Processor -- start 2024-03-29T04:10:03.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.381+0000 7f67c1b9d640 1 -- start start 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c1b9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 0x7f67bc0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c1b9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67bc06d7b0 0x7f67bc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c1b9d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67bc0cc640 con 0x7f67bc06c8a0 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c1b9d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f67bc06cd00 con 0x7f67bc06d7b0 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c1b9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67bc15eab0 con 0x7f67bc15e6d0 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c139c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67bc06d7b0 0x7f67bc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.362 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67bbfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f67bc06c8a0 0x7f67bc06d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48694/0 (socket says 172.21.15.181:48694) 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67bbfff640 1 -- 172.21.15.181:0/4209954094 learned_addr learned my addr 172.21.15.181:0/4209954094 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c0b9b640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 0x7f67bc0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c139c640 1 -- 172.21.15.181:0/4209954094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 msgr2=0x7f67bc0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c139c640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 0x7f67bc0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c139c640 1 -- 172.21.15.181:0/4209954094 >> v1:172.21.15.181:6789/0 conn(0x7f67bc06c8a0 legacy=0x7f67bc06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67bb7fe640 1 -- 172.21.15.181:0/4209954094 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325137668 0 0) 0x7f67bc0cc640 con 0x7f67bc06c8a0 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c139c640 1 -- 172.21.15.181:0/4209954094 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc0d0fe0 con 0x7f67bc06d7b0 2024-03-29T04:10:03.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.382+0000 7f67c0b9b640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 0x7f67bc0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:03.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.383+0000 7f67c139c640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0ccdd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009d90 tx=0x7f67b402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ac33ea0c5b1beb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.383+0000 7f67bb7fe640 1 -- 172.21.15.181:0/4209954094 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4039070 con 0x7f67bc06d7b0 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.383+0000 7f67bb7fe640 1 -- 172.21.15.181:0/4209954094 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b402cbc0 con 0x7f67bc06d7b0 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.383+0000 7f67bb7fe640 1 -- 172.21.15.181:0/4209954094 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4034440 con 0x7f67bc06d7b0 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 -- 172.21.15.181:0/4209954094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 msgr2=0x7f67bc0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0ccdd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009d90 tx=0x7f67b402be10 comp rx=0 tx=0).stop 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 -- 172.21.15.181:0/4209954094 shutdown_connections 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0ccdd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/4209954094 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc15e6d0 0x7f67bc0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 -- 172.21.15.181:0/4209954094 >> 172.21.15.181:0/4209954094 conn(0x7f67bc09aa30 msgr2=0x7f67bc09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 -- 172.21.15.181:0/4209954094 shutdown_connections 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.384+0000 7f67c1b9d640 1 -- 172.21.15.181:0/4209954094 wait complete. 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.385+0000 7f67c1b9d640 1 Processor -- start 2024-03-29T04:10:03.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.385+0000 7f67c1b9d640 1 -- start start 2024-03-29T04:10:03.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc06c8a0 0x7f67bc0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67bc15e6d0 0x7f67bc1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc0cc940 con 0x7f67bc06d7b0 2024-03-29T04:10:03.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc0d0d90 con 0x7f67bc15e6d0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c1b9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67bc0d15a0 con 0x7f67bc06c8a0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44770/0 (socket says 172.21.15.181:44770) 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 -- 172.21.15.181:0/2816447695 learned_addr learned my addr 172.21.15.181:0/2816447695 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c0b9b640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc06c8a0 0x7f67bc0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67c139c640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67bc15e6d0 0x7f67bc1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 -- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc06c8a0 msgr2=0x7f67bc0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc06c8a0 0x7f67bc0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 -- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67bc15e6d0 msgr2=0x7f67bc1f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67bc15e6d0 0x7f67bc1f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 -- 172.21.15.181:0/2816447695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc14f360 con 0x7f67bc06d7b0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.386+0000 7f67bbfff640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c4d0 tx=0x7f67b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.387+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0018070 con 0x7f67bc06d7b0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.387+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b0007500 con 0x7f67bc06d7b0 2024-03-29T04:10:03.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.387+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc14f640 con 0x7f67bc06d7b0 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.387+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004a20 con 0x7f67bc06d7b0 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.387+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc14fc00 con 0x7f67bc06d7b0 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.388+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc075890 con 0x7f67bc06d7b0 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.388+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f67b001b070 con 0x7f67bc06d7b0 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.388+0000 7f67b97fa640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 0x7f67a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.389+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f67b008acc0 con 0x7f67bc06d7b0 2024-03-29T04:10:03.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.389+0000 7f67c0b9b640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 0x7f67a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.389+0000 7f67c0b9b640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 0x7f67a806ea20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009560 tx=0x7f67ac009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.393+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f67b0058920 con 0x7f67bc06d7b0 2024-03-29T04:10:03.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.569+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67bc06ce80 con 0x7f67bc06d7b0 2024-03-29T04:10:03.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.569+0000 7f67b97fa640 1 -- 172.21.15.181:0/2816447695 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f67b005cc20 con 0x7f67bc06d7b0 2024-03-29T04:10:03.549 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 msgr2=0x7f67a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 0x7f67a806ea20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009560 tx=0x7f67ac009190 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 msgr2=0x7f67bc0b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c4d0 tx=0x7f67b000c990 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 shutdown_connections 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67bc15e6d0 0x7f67bc1f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67bc06d7b0 0x7f67bc0b67b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f67a806c560 0x7f67a806ea20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 --2- 172.21.15.181:0/2816447695 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67bc06c8a0 0x7f67bc0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 >> 172.21.15.181:0/2816447695 conn(0x7f67bc09aa30 msgr2=0x7f67bc0c9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 shutdown_connections 2024-03-29T04:10:03.552 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.571+0000 7f67c1b9d640 1 -- 172.21.15.181:0/2816447695 wait complete. 2024-03-29T04:10:03.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:03.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.683+0000 7f361c1f2640 1 Processor -- start 2024-03-29T04:10:03.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.683+0000 7f361c1f2640 1 -- start start 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361c1f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361415e7a0 0x7f36140cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361c1f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36140cec60 0x7f36140d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361c1f2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36140d3120 con 0x7f36140d3ee0 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361c1f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f36140cfc40 con 0x7f36140cec60 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361c1f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36140d3400 con 0x7f361415e7a0 2024-03-29T04:10:03.664 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361978d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36140cec60 0x7f36140d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361978d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36140cec60 0x7f36140d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44782/0 (socket says 172.21.15.181:44782) 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f361978d640 1 -- 172.21.15.181:0/2624985155 learned_addr learned my addr 172.21.15.181:0/2624985155 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.684+0000 7f3619f8e640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361415e7a0 0x7f36140cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f361978d640 1 -- 172.21.15.181:0/2624985155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361415e7a0 msgr2=0x7f36140cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f361978d640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361415e7a0 0x7f36140cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f3618f8c640 1 -- 172.21.15.181:0/2624985155 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165121721 0 0) 0x7f36140d3120 con 0x7f36140d3ee0 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f361978d640 1 -- 172.21.15.181:0/2624985155 >> v1:172.21.15.187:6789/0 conn(0x7f36140d3ee0 legacy=0x7f36140b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f361978d640 1 -- 172.21.15.181:0/2624985155 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36140d36e0 con 0x7f36140cec60 2024-03-29T04:10:03.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.685+0000 7f361978d640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140d12b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3604009870 tx=0x7f360402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc224f2870706ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f3618f8c640 1 -- 172.21.15.181:0/2624985155 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604035020 con 0x7f36140cec60 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f3618f8c640 1 -- 172.21.15.181:0/2624985155 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604031640 con 0x7f36140cec60 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f3618f8c640 1 -- 172.21.15.181:0/2624985155 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360402d040 con 0x7f36140cec60 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 -- 172.21.15.181:0/2624985155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 msgr2=0x7f36140d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140d12b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3604009870 tx=0x7f360402b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 -- 172.21.15.181:0/2624985155 shutdown_connections 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140d12b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2624985155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f361415e7a0 0x7f36140cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 -- 172.21.15.181:0/2624985155 >> 172.21.15.181:0/2624985155 conn(0x7f36140676b0 msgr2=0x7f3614067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 -- 172.21.15.181:0/2624985155 shutdown_connections 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.686+0000 7f361c1f2640 1 -- 172.21.15.181:0/2624985155 wait complete. 2024-03-29T04:10:03.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.687+0000 7f361c1f2640 1 Processor -- start 2024-03-29T04:10:03.668 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.687+0000 7f361c1f2640 1 -- start start 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36140d3ee0 0x7f36140b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36140b8c20 0x7f36140b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36140d17f0 con 0x7f36140cec60 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36140cf930 con 0x7f36140b8c20 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361c1f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36140d1970 con 0x7f36140d3ee0 2024-03-29T04:10:03.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f3619f8e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f3619f8e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44798/0 (socket says 172.21.15.181:44798) 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f3619f8e640 1 -- 172.21.15.181:0/2769703284 learned_addr learned my addr 172.21.15.181:0/2769703284 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361a78f640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36140b8c20 0x7f36140b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.688+0000 7f361978d640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36140d3ee0 0x7f36140b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 -- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36140d3ee0 msgr2=0x7f36140b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36140d3ee0 0x7f36140b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 -- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36140b8c20 msgr2=0x7f36140b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36140b8c20 0x7f36140b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 -- 172.21.15.181:0/2769703284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36140b4690 con 0x7f36140cec60 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f3619f8e640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f361000efc0 tx=0x7f361000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610018020 con 0x7f36140cec60 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3610014480 con 0x7f36140cec60 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36140b4970 con 0x7f36140cec60 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610014850 con 0x7f36140cec60 2024-03-29T04:10:03.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.689+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36140b4f30 con 0x7f36140cec60 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.690+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36140b9030 con 0x7f36140cec60 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.690+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3610005d10 con 0x7f36140cec60 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.691+0000 7f360a7fc640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 0x7f35f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.691+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f361008c730 con 0x7f36140cec60 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.691+0000 7f361978d640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 0x7f35f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:03.672 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.691+0000 7f361978d640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 0x7f35f806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3604036000 tx=0x7f360403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:03.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.695+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36140b9030 con 0x7f36140cec60 2024-03-29T04:10:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.882+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3614075900 con 0x7f36140cec60 2024-03-29T04:10:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.882+0000 7f360a7fc640 1 -- 172.21.15.181:0/2769703284 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f3610022030 con 0x7f36140cec60 2024-03-29T04:10:03.861 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 msgr2=0x7f35f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 0x7f35f806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3604036000 tx=0x7f360403c040 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 msgr2=0x7f36140b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f361000efc0 tx=0x7f361000c490 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 shutdown_connections 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36140b8c20 0x7f36140b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f35f806c530 0x7f35f806e9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36140d3ee0 0x7f36140b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 --2- 172.21.15.181:0/2769703284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36140cec60 0x7f36140b27a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 >> 172.21.15.181:0/2769703284 conn(0x7f36140676b0 msgr2=0x7f36140c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:03.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 shutdown_connections 2024-03-29T04:10:03.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:03.884+0000 7f361c1f2640 1 -- 172.21.15.181:0/2769703284 wait complete. 2024-03-29T04:10:03.874 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-29T04:10:03.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:03.875 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-29T04:10:03.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:10:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:03.904 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:10:03.962 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:10:03.962 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:03.978 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:10:03.978 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:04.033 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:10:04.034 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:04.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:04.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.250+0000 7f3638cd3640 1 Processor -- start 2024-03-29T04:10:04.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.250+0000 7f3638cd3640 1 -- start start 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3638cd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d4a90 0x7f36340d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3638cd3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36340d2f00 0x7f36340b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3638cd3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36340d36f0 con 0x7f363415e950 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3638cd3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f36340d3cb0 con 0x7f36340d2f00 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3638cd3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36340d39d0 con 0x7f36340d4a90 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3632d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36340d2f00 0x7f36340b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3632d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36340d2f00 0x7f36340b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44800/0 (socket says 172.21.15.181:44800) 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3632d9d640 1 -- 172.21.15.181:0/4012164871 learned_addr learned my addr 172.21.15.181:0/4012164871 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.251+0000 7f3631d9b640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d4a90 0x7f36340d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.252+0000 7f3632d9d640 1 -- 172.21.15.181:0/4012164871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d4a90 msgr2=0x7f36340d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.252+0000 7f3632d9d640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d4a90 0x7f36340d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.252+0000 7f3632d9d640 1 -- 172.21.15.181:0/4012164871 >> v1:172.21.15.187:6789/0 conn(0x7f363415e950 legacy=0x7f36340d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.232 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.252+0000 7f3632d9d640 1 -- 172.21.15.181:0/4012164871 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36340b7ac0 con 0x7f36340d2f00 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.252+0000 7f3632d9d640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36340d2f00 0x7f36340b82c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f362c009870 tx=0x7f362c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17a472ddfc8e721 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f363159a640 1 -- 172.21.15.181:0/4012164871 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c02f840 con 0x7f36340d2f00 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f363159a640 1 -- 172.21.15.181:0/4012164871 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c02f9e0 con 0x7f36340d2f00 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f363159a640 1 -- 172.21.15.181:0/4012164871 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c02fdb0 con 0x7f36340d2f00 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 -- 172.21.15.181:0/4012164871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36340d2f00 msgr2=0x7f36340b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36340d2f00 0x7f36340b82c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f362c009870 tx=0x7f362c02ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 -- 172.21.15.181:0/4012164871 shutdown_connections 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36340d2f00 0x7f36340b82c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 --2- 172.21.15.181:0/4012164871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d4a90 0x7f36340d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 -- 172.21.15.181:0/4012164871 >> 172.21.15.181:0/4012164871 conn(0x7f36340676e0 msgr2=0x7f3634067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 -- 172.21.15.181:0/4012164871 shutdown_connections 2024-03-29T04:10:04.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.253+0000 7f3638cd3640 1 -- 172.21.15.181:0/4012164871 wait complete. 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.254+0000 7f3638cd3640 1 Processor -- start 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.254+0000 7f3638cd3640 1 -- start start 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d2f00 0x7f3634154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36340d4a90 0x7f36340aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f363415ed30 con 0x7f363415e950 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36340d33b0 con 0x7f36340d4a90 2024-03-29T04:10:04.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3638cd3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f363415eeb0 con 0x7f36340d2f00 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3632d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3632d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44806/0 (socket says 172.21.15.181:44806) 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3632d9d640 1 -- 172.21.15.181:0/2065586722 learned_addr learned my addr 172.21.15.181:0/2065586722 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f3631d9b640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36340d4a90 0x7f36340aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.255+0000 7f363259c640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d2f00 0x7f3634154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 -- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d2f00 msgr2=0x7f3634154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d2f00 0x7f3634154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 -- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36340d4a90 msgr2=0x7f36340aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36340d4a90 0x7f36340aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 -- 172.21.15.181:0/2065586722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36340c8a60 con 0x7f363415e950 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3632d9d640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f362c009870 tx=0x7f362c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c037c40 con 0x7f363415e950 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c046070 con 0x7f363415e950 2024-03-29T04:10:04.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36340c8d40 con 0x7f363415e950 2024-03-29T04:10:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c041400 con 0x7f363415e950 2024-03-29T04:10:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.256+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36340c9300 con 0x7f363415e950 2024-03-29T04:10:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.257+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36340c9500 con 0x7f363415e950 2024-03-29T04:10:04.238 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.257+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f362c040070 con 0x7f363415e950 2024-03-29T04:10:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.258+0000 7f3622ffd640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 0x7f361406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.258+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f362c0aded0 con 0x7f363415e950 2024-03-29T04:10:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.258+0000 7f363259c640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 0x7f361406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.239 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.258+0000 7f363259c640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 0x7f361406e9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3624001f10 tx=0x7f3624009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.241 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.262+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36340c9500 con 0x7f363415e950 2024-03-29T04:10:04.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.441+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36340b9770 con 0x7f363415e950 2024-03-29T04:10:04.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.443+0000 7f3622ffd640 1 -- 172.21.15.181:0/2065586722 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7f362c033030 con 0x7f363415e950 2024-03-29T04:10:04.423 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:04.423 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:04.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.444+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 msgr2=0x7f361406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.444+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 0x7f361406e9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3624001f10 tx=0x7f3624009040 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 msgr2=0x7f3634161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f362c009870 tx=0x7f362c004730 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 shutdown_connections 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363415e950 0x7f3634161280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36340d4a90 0x7f36340aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f361406c530 0x7f361406e9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 --2- 172.21.15.181:0/2065586722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36340d2f00 0x7f3634154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 >> 172.21.15.181:0/2065586722 conn(0x7f36340676e0 msgr2=0x7f36340cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 shutdown_connections 2024-03-29T04:10:04.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.445+0000 7f3638cd3640 1 -- 172.21.15.181:0/2065586722 wait complete. 2024-03-29T04:10:04.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:04.436 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 Processor -- start 2024-03-29T04:10:04.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 -- start start 2024-03-29T04:10:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd0a00b2f20 0x7fd0a015f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00b35e0 con 0x7fd0a0072d50 2024-03-29T04:10:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00b38c0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.560+0000 7fd0a5a88640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a00b3e80 con 0x7fd0a00b2f20 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09effd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd0a0072d50 0x7fd0a00ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48700/0 (socket says 172.21.15.181:48700) 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09effd640 1 -- 172.21.15.181:0/4193034551 learned_addr learned my addr 172.21.15.181:0/4193034551 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09ffff640 1 --2- 172.21.15.181:0/4193034551 >> v2:172.21.15.187:3300/0 conn(0x7fd0a00b2f20 0x7fd0a015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419876267 0 0) 0x7fd0a00b35e0 con 0x7fd0a0072d50 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd088003520 con 0x7fd0a0072d50 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251064692 0 0) 0x7fd0a00b38c0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a00b35e0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09ffff640 1 -- 172.21.15.181:0/4193034551 >> v1:172.21.15.187:6789/0 conn(0x7fd0a0144cb0 legacy=0x7fd0a0072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09ffff640 1 -- 172.21.15.181:0/4193034551 >> v1:172.21.15.181:6789/0 conn(0x7fd0a0072d50 legacy=0x7fd0a00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.561+0000 7fd09ffff640 1 -- 172.21.15.181:0/4193034551 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00b3ba0 con 0x7fd0a00b2f20 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd09ffff640 1 --2- 172.21.15.181:0/4193034551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015f800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a57c89de9435b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098039070 con 0x7fd0a00b2f20 2024-03-29T04:10:04.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09802cd00 con 0x7fd0a00b2f20 2024-03-29T04:10:04.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd09e7fc640 1 -- 172.21.15.181:0/4193034551 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098034640 con 0x7fd0a00b2f20 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 -- 172.21.15.181:0/4193034551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 msgr2=0x7fd0a015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/4193034551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015f800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802bf50 comp rx=0 tx=0).stop 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 -- 172.21.15.181:0/4193034551 shutdown_connections 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/4193034551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015f800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 -- 172.21.15.181:0/4193034551 >> 172.21.15.181:0/4193034551 conn(0x7fd0a00b5b00 msgr2=0x7fd0a006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 -- 172.21.15.181:0/4193034551 shutdown_connections 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.562+0000 7fd0a5a88640 1 -- 172.21.15.181:0/4193034551 wait complete. 2024-03-29T04:10:04.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 Processor -- start 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 -- start start 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 0x7fd0a01f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00ce760 con 0x7fd0a0144cb0 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00b3390 con 0x7fd0a00b2f20 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.564+0000 7fd0a5a88640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00ce8e0 con 0x7fd0a0072d50 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44820/0 (socket says 172.21.15.181:44820) 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 -- 172.21.15.181:0/1848640252 learned_addr learned my addr 172.21.15.181:0/1848640252 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09f7fe640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 0x7fd0a01f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09effd640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 -- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 msgr2=0x7fd0a01f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 0x7fd0a01f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 -- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 msgr2=0x7fd0a015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 -- 172.21.15.181:0/1848640252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a0150080 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09f7fe640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 0x7fd0a01f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09effd640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd09ffff640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098039050 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098045070 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a013b5d0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098009450 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.565+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a015d1c0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.566+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0075890 con 0x7fd0a0144cb0 2024-03-29T04:10:04.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.567+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd09803e830 con 0x7fd0a0144cb0 2024-03-29T04:10:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.567+0000 7fd087fff640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.567+0000 7fd09f7fe640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.567+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd0980ae6d0 con 0x7fd0a0144cb0 2024-03-29T04:10:04.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.568+0000 7fd09f7fe640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 0x7fd08c06e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd090007fd0 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.551 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.572+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd0a0150080 con 0x7fd0a0144cb0 2024-03-29T04:10:04.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.745+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0a00b4030 con 0x7fd0a0144cb0 2024-03-29T04:10:04.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.745+0000 7fd087fff640 1 -- 172.21.15.181:0/1848640252 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fd09807c480 con 0x7fd0a0144cb0 2024-03-29T04:10:04.725 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 msgr2=0x7fd08c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 0x7fd08c06e9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd090007fd0 tx=0x7fd090009040 comp rx=0 tx=0).stop 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 msgr2=0x7fd0a015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd098004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 shutdown_connections 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a0144cb0 0x7fd0a015dc80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0a00b2f20 0x7fd0a015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 --2- 172.21.15.181:0/1848640252 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0a0072d50 0x7fd0a01f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.747+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 >> 172.21.15.181:0/1848640252 conn(0x7fd0a00b5b00 msgr2=0x7fd0a006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.748+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 shutdown_connections 2024-03-29T04:10:04.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.748+0000 7fd0a5a88640 1 -- 172.21.15.181:0/1848640252 wait complete. 2024-03-29T04:10:04.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:04.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.861+0000 7f52d297c640 1 Processor -- start 2024-03-29T04:10:04.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.861+0000 7f52d297c640 1 -- start start 2024-03-29T04:10:04.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d297c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f52cc144cb0 0x7f52cc0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d297c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52cc15f800 0x7f52cc072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d297c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc0b35e0 con 0x7f52cc072b80 2024-03-29T04:10:04.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d297c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc072fd0 con 0x7f52cc15f800 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d297c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc0b3e80 con 0x7f52cc144cb0 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d0f19640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f52cc072b80 0x7f52cc071420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48710/0 (socket says 172.21.15.181:48710) 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52d0f19640 1 -- 172.21.15.181:0/1260375856 learned_addr learned my addr 172.21.15.181:0/1260375856 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52cb7fe640 1 --2- 172.21.15.181:0/1260375856 >> v2:172.21.15.181:3300/0 conn(0x7f52cc15f800 0x7f52cc072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.862+0000 7f52cbfff640 1 --2- 172.21.15.181:0/1260375856 >> v2:172.21.15.187:3300/0 conn(0x7f52cc144cb0 0x7f52cc0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52caffd640 1 -- 172.21.15.181:0/1260375856 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517287054 0 0) 0x7f52cc0b35e0 con 0x7f52cc072b80 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52caffd640 1 -- 172.21.15.181:0/1260375856 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b4003520 con 0x7f52cc072b80 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52cbfff640 1 -- 172.21.15.181:0/1260375856 >> v2:172.21.15.181:3300/0 conn(0x7f52cc15f800 msgr2=0x7f52cc072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52cbfff640 1 --2- 172.21.15.181:0/1260375856 >> v2:172.21.15.181:3300/0 conn(0x7f52cc15f800 0x7f52cc072640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52cbfff640 1 -- 172.21.15.181:0/1260375856 >> v1:172.21.15.181:6789/0 conn(0x7f52cc072b80 legacy=0x7f52cc071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52cbfff640 1 -- 172.21.15.181:0/1260375856 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0b38c0 con 0x7f52cc144cb0 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.863+0000 7f52cbfff640 1 --2- 172.21.15.181:0/1260375856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc0b2f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52cc06e3e0 tx=0x7f52bc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883e448bbf158a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52caffd640 1 -- 172.21.15.181:0/1260375856 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035020 con 0x7f52cc144cb0 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52caffd640 1 -- 172.21.15.181:0/1260375856 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc031830 con 0x7f52cc144cb0 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52caffd640 1 -- 172.21.15.181:0/1260375856 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc031c00 con 0x7f52cc144cb0 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 -- 172.21.15.181:0/1260375856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 msgr2=0x7f52cc0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 --2- 172.21.15.181:0/1260375856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc0b2f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52cc06e3e0 tx=0x7f52bc02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 -- 172.21.15.181:0/1260375856 shutdown_connections 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 --2- 172.21.15.181:0/1260375856 >> v2:172.21.15.181:3300/0 conn(0x7f52cc15f800 0x7f52cc072640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 --2- 172.21.15.181:0/1260375856 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc0b2f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 -- 172.21.15.181:0/1260375856 >> 172.21.15.181:0/1260375856 conn(0x7f52cc0b5b00 msgr2=0x7f52cc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 -- 172.21.15.181:0/1260375856 shutdown_connections 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.864+0000 7f52d297c640 1 -- 172.21.15.181:0/1260375856 wait complete. 2024-03-29T04:10:04.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.865+0000 7f52d297c640 1 Processor -- start 2024-03-29T04:10:04.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 -- start start 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc072b80 0x7f52cc1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc15fc40 con 0x7f52cc15f800 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0cdb30 con 0x7f52cc144cb0 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d297c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc0cf060 con 0x7f52cc072b80 2024-03-29T04:10:04.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d0f19640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d0f19640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44832/0 (socket says 172.21.15.181:44832) 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52d0f19640 1 -- 172.21.15.181:0/2377721545 learned_addr learned my addr 172.21.15.181:0/2377721545 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52cbfff640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc072b80 0x7f52cc1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.866+0000 7f52cb7fe640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 -- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc072b80 msgr2=0x7f52cc1e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc072b80 0x7f52cc1e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 -- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 msgr2=0x7f52cc1e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc1e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 -- 172.21.15.181:0/2377721545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc12c510 con 0x7f52cc15f800 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52cb7fe640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d0f19640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f52c400cb40 tx=0x7f52c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4017070 con 0x7f52cc15f800 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c400aca0 con 0x7f52cc15f800 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc12c7f0 con 0x7f52cc15f800 2024-03-29T04:10:04.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012620 con 0x7f52cc15f800 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.867+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc12cdb0 con 0x7f52cc15f800 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.868+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc075890 con 0x7f52cc15f800 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.868+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52c401a070 con 0x7f52cc15f800 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.869+0000 7f52c8ff9640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.869+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f52c408b910 con 0x7f52cc15f800 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.869+0000 7f52cbfff640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:04.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.869+0000 7f52cbfff640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 0x7f52b806e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f52bc036000 tx=0x7f52bc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:04.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:04.873+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f52c40594f0 con 0x7f52cc15f800 2024-03-29T04:10:05.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.046+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52cc075130 con 0x7f52cc15f800 2024-03-29T04:10:05.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.047+0000 7f52c8ff9640 1 -- 172.21.15.181:0/2377721545 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f52c405d7f0 con 0x7f52cc15f800 2024-03-29T04:10:05.026 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 msgr2=0x7f52b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 0x7f52b806e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f52bc036000 tx=0x7f52bc008030 comp rx=0 tx=0).stop 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 msgr2=0x7f52cc15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f52c400cb40 tx=0x7f52c400a060 comp rx=0 tx=0).stop 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 shutdown_connections 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52cc15f800 0x7f52cc15a730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc144cb0 0x7f52cc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52b806c530 0x7f52b806e9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.048+0000 7f52d297c640 1 --2- 172.21.15.181:0/2377721545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc072b80 0x7f52cc1e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.049+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 >> 172.21.15.181:0/2377721545 conn(0x7f52cc0b5b00 msgr2=0x7f52cc071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.049+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 shutdown_connections 2024-03-29T04:10:05.029 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.049+0000 7f52d297c640 1 -- 172.21.15.181:0/2377721545 wait complete. 2024-03-29T04:10:05.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:05.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.162+0000 7fc6b90a7640 1 Processor -- start 2024-03-29T04:10:05.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b90a7640 1 -- start start 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b90a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 0x7fc6b406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b90a7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b40b0b80 con 0x7fc6b40b0730 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b90a7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b4072560 con 0x7fc6b406bac0 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b90a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b406bf20 con 0x7fc6b415c2d0 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.163+0000 7fc6b359e640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc6b40b0730 0x7fc6b406f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48718/0 (socket says 172.21.15.181:48718) 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b359e640 1 -- 172.21.15.181:0/2177621220 learned_addr learned my addr 172.21.15.181:0/2177621220 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:05.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b2d9d640 1 --2- 172.21.15.181:0/2177621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 0x7fc6b406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989131570 0 0) 0x7fc6b40b0b80 con 0x7fc6b40b0730 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc69c003520 con 0x7fc6b40b0730 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393281552 0 0) 0x7fc6b4072560 con 0x7fc6b406bac0 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/2177621220 >> v1:172.21.15.187:6789/0 conn(0x7fc6b406bac0 legacy=0x7fc6b40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/2177621220 >> v1:172.21.15.181:6789/0 conn(0x7fc6b40b0730 legacy=0x7fc6b406f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.164+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/2177621220 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b4072840 con 0x7fc6b415c2d0 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b2d9d640 1 --2- 172.21.15.181:0/2177621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 0x7fc6b406b580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40097d0 tx=0x7fc6a402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359ff7f8dc22d348 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4035020 con 0x7fc6b415c2d0 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a4031890 con 0x7fc6b415c2d0 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b1d9b640 1 -- 172.21.15.181:0/2177621220 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4031c40 con 0x7fc6b415c2d0 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 -- 172.21.15.181:0/2177621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 msgr2=0x7fc6b406b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/2177621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 0x7fc6b406b580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40097d0 tx=0x7fc6a402b930 comp rx=0 tx=0).stop 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 -- 172.21.15.181:0/2177621220 shutdown_connections 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/2177621220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b415c2d0 0x7fc6b406b580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 -- 172.21.15.181:0/2177621220 >> 172.21.15.181:0/2177621220 conn(0x7fc6b40d1160 msgr2=0x7fc6b40d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 -- 172.21.15.181:0/2177621220 shutdown_connections 2024-03-29T04:10:05.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.165+0000 7fc6b90a7640 1 -- 172.21.15.181:0/2177621220 wait complete. 2024-03-29T04:10:05.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.167+0000 7fc6b90a7640 1 Processor -- start 2024-03-29T04:10:05.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.167+0000 7fc6b90a7640 1 -- start start 2024-03-29T04:10:05.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b40b0730 0x7fc6b40b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 0x7fc6b41f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4072e00 con 0x7fc6b415c2d0 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4073170 con 0x7fc6b406bac0 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b4073600 con 0x7fc6b40b0730 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 0x7fc6b41f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 0x7fc6b41f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44846/0 (socket says 172.21.15.181:44846) 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:43422/0 (socket says 172.21.15.181:43422) 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b359e640 1 -- 172.21.15.181:0/622824735 learned_addr learned my addr 172.21.15.181:0/622824735 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b259c640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b40b0730 0x7fc6b40b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b40b0730 msgr2=0x7fc6b40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b40b0730 0x7fc6b40b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 msgr2=0x7fc6b41f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 0x7fc6b41f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 -- 172.21.15.181:0/622824735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b415b770 con 0x7fc6b406bac0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b2d9d640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4033000 tx=0x7fc6a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4040070 con 0x7fc6b406bac0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a4030d40 con 0x7fc6b406bac0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b415b490 con 0x7fc6b406bac0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4035040 con 0x7fc6b406bac0 2024-03-29T04:10:05.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.168+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b40af830 con 0x7fc6b406bac0 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.169+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b415e490 con 0x7fc6b406bac0 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.170+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc6a404b060 con 0x7fc6b406bac0 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.170+0000 7fc69b7fe640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 0x7fc6a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.170+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc6a40ad930 con 0x7fc6b406bac0 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.170+0000 7fc6b259c640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 0x7fc6a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.171+0000 7fc6b259c640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 0x7fc6a006e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.154 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.175+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc6b415e490 con 0x7fc6b406bac0 2024-03-29T04:10:05.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.349+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6b4075890 con 0x7fc6b406bac0 2024-03-29T04:10:05.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.349+0000 7fc69b7fe640 1 -- 172.21.15.181:0/622824735 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc6a40abe20 con 0x7fc6b406bac0 2024-03-29T04:10:05.329 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 msgr2=0x7fc6a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 0x7fc6a006e9f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8009040 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 msgr2=0x7fc6b40ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4033000 tx=0x7fc6a40046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 shutdown_connections 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6b415c2d0 0x7fc6b41f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc6a006c530 0x7fc6a006e9f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6b40b0730 0x7fc6b40b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 --2- 172.21.15.181:0/622824735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6b406bac0 0x7fc6b40ae2b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 >> 172.21.15.181:0/622824735 conn(0x7fc6b40d1160 msgr2=0x7fc6b4160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 shutdown_connections 2024-03-29T04:10:05.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.351+0000 7fc6b90a7640 1 -- 172.21.15.181:0/622824735 wait complete. 2024-03-29T04:10:05.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:05.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.467+0000 7f8e0c6fb640 1 Processor -- start 2024-03-29T04:10:05.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.467+0000 7f8e0c6fb640 1 -- start start 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0c6fb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8e040d13d0 0x7f8e040cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0c6fb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e040d3ba0 0x7f8e040b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0c6fb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e040cf860 con 0x7f8e040b3760 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0c6fb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8e040cfb40 con 0x7f8e040d3ba0 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0c6fb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8e040d33a0 con 0x7f8e040d13d0 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e040d3ba0 0x7f8e040b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e040d3ba0 0x7f8e040b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44862/0 (socket says 172.21.15.181:44862) 2024-03-29T04:10:05.448 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2154935864 learned_addr learned my addr 172.21.15.181:0/2154935864 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:05.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0a497640 1 --2- 172.21.15.181:0/2154935864 >> v2:172.21.15.187:3300/0 conn(0x7f8e040d13d0 0x7f8e040cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2154935864 >> v2:172.21.15.187:3300/0 conn(0x7f8e040d13d0 msgr2=0x7f8e040cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:05.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 --2- 172.21.15.181:0/2154935864 >> v2:172.21.15.187:3300/0 conn(0x7f8e040d13d0 0x7f8e040cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2154935864 >> v1:172.21.15.187:6789/0 conn(0x7f8e040b3760 legacy=0x7f8e040d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.468+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2154935864 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e040d3680 con 0x7f8e040d3ba0 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.469+0000 7f8e0ac98640 1 --2- 172.21.15.181:0/2154935864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e040b7740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e0406c8c0 tx=0x7f8dfc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c5bea66fbbad09 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.469+0000 7f8e09495640 1 -- 172.21.15.181:0/2154935864 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc02cb20 con 0x7f8e040d3ba0 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.469+0000 7f8e09495640 1 -- 172.21.15.181:0/2154935864 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc02ccc0 con 0x7f8e040d3ba0 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.469+0000 7f8e09495640 1 -- 172.21.15.181:0/2154935864 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc034610 con 0x7f8e040d3ba0 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2154935864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 msgr2=0x7f8e040b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2154935864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e040b7740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e0406c8c0 tx=0x7f8dfc02bf80 comp rx=0 tx=0).stop 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2154935864 shutdown_connections 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2154935864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e040b7740 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2154935864 >> v2:172.21.15.187:3300/0 conn(0x7f8e040d13d0 0x7f8e040cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2154935864 >> 172.21.15.181:0/2154935864 conn(0x7f8e04067710 msgr2=0x7f8e04067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2154935864 shutdown_connections 2024-03-29T04:10:05.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.470+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2154935864 wait complete. 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.471+0000 7f8e0c6fb640 1 Processor -- start 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.471+0000 7f8e0c6fb640 1 -- start start 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e040b3760 0x7f8e0414a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e040d13d0 0x7f8e0414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0415eab0 con 0x7f8e040d3ba0 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0415e930 con 0x7f8e040d13d0 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0c6fb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8e040cf4c0 con 0x7f8e040b3760 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44878/0 (socket says 172.21.15.181:44878) 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2778243355 learned_addr learned my addr 172.21.15.181:0/2778243355 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e09c96640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e040d13d0 0x7f8e0414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0a497640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e040b3760 0x7f8e0414a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e040b3760 msgr2=0x7f8e0414a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e040b3760 0x7f8e0414a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e040d13d0 msgr2=0x7f8e0414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e040d13d0 0x7f8e0414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 -- 172.21.15.181:0/2778243355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e041ea6b0 con 0x7f8e040d3ba0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.472+0000 7f8e0ac98640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009f40 tx=0x7f8dfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.473+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc043020 con 0x7f8e040d3ba0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.473+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc03e450 con 0x7f8e040d3ba0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.473+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0414ac10 con 0x7f8e040d3ba0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.473+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc009510 con 0x7f8e040d3ba0 2024-03-29T04:10:05.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.473+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0414b1d0 con 0x7f8e040d3ba0 2024-03-29T04:10:05.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.474+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e04075890 con 0x7f8e040d3ba0 2024-03-29T04:10:05.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.474+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8dfc046020 con 0x7f8e040d3ba0 2024-03-29T04:10:05.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.474+0000 7f8dfaffd640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 0x7f8de806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.474+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8dfc0ae140 con 0x7f8e040d3ba0 2024-03-29T04:10:05.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.475+0000 7f8e0a497640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 0x7f8de806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.475+0000 7f8e0a497640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 0x7f8de806e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009b40 tx=0x7f8e00009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.479+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8dfc04d030 con 0x7f8e040d3ba0 2024-03-29T04:10:05.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.654+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8e04161f20 con 0x7f8e040d3ba0 2024-03-29T04:10:05.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.655+0000 7f8dfaffd640 1 -- 172.21.15.181:0/2778243355 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f8dfc07bd10 con 0x7f8e040d3ba0 2024-03-29T04:10:05.634 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:05.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.656+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 msgr2=0x7f8de806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.656+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 0x7f8de806e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009b40 tx=0x7f8e00009190 comp rx=0 tx=0).stop 2024-03-29T04:10:05.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.656+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 msgr2=0x7f8e041f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.656+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009f40 tx=0x7f8dfc004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 shutdown_connections 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e040d3ba0 0x7f8e041f3b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e040d13d0 0x7f8e0414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8de806c530 0x7f8de806e9f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 --2- 172.21.15.181:0/2778243355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e040b3760 0x7f8e0414a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 >> 172.21.15.181:0/2778243355 conn(0x7f8e04067710 msgr2=0x7f8e040d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 shutdown_connections 2024-03-29T04:10:05.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.657+0000 7f8e0c6fb640 1 -- 172.21.15.181:0/2778243355 wait complete. 2024-03-29T04:10:05.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.768+0000 7fc4fbeeb640 1 Processor -- start 2024-03-29T04:10:05.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.768+0000 7fc4fbeeb640 1 -- start start 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4f415a990 0x7fc4f4071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 0x7fc4f40cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 0x7fc4f40cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f4072ad0 con 0x7fc4f415a990 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f4073700 con 0x7fc4f415ad70 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fbeeb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f4073000 con 0x7fc4f40ce160 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4f415a990 0x7fc4f4071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc4f415a990 0x7fc4f4071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44890/0 (socket says 172.21.15.181:44890) 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9486640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 0x7fc4f40cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 -- 172.21.15.181:0/2346315750 learned_addr learned my addr 172.21.15.181:0/2346315750 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:05.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fa488640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 0x7fc4f40cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 -- 172.21.15.181:0/2346315750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 msgr2=0x7fc4f40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 0x7fc4f40cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 -- 172.21.15.181:0/2346315750 >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 msgr2=0x7fc4f40cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/2346315750 >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 0x7fc4f40cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9c87640 1 -- 172.21.15.181:0/2346315750 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4072d20 con 0x7fc4f415a990 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4fa488640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 0x7fc4f40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.769+0000 7fc4f9486640 1 --2- 172.21.15.181:0/2346315750 >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 0x7fc4f40cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.770+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f415a990 0x7fc4f4071ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40c3b50 tx=0x7fc4f002dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f65ffeba59bdc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.770+0000 7fc4f8c85640 1 -- 172.21.15.181:0/2346315750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f003b070 con 0x7fc4f415a990 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.770+0000 7fc4f8c85640 1 -- 172.21.15.181:0/2346315750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f002ed70 con 0x7fc4f415a990 2024-03-29T04:10:05.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.770+0000 7fc4f8c85640 1 -- 172.21.15.181:0/2346315750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0036660 con 0x7fc4f415a990 2024-03-29T04:10:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/2346315750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f415a990 msgr2=0x7fc4f4071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f415a990 0x7fc4f4071ab0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40c3b50 tx=0x7fc4f002dfc0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/2346315750 shutdown_connections 2024-03-29T04:10:05.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f40ce160 0x7fc4f40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/2346315750 >> v2:172.21.15.187:3300/0 conn(0x7fc4f415ad70 0x7fc4f40cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/2346315750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f415a990 0x7fc4f4071ab0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/2346315750 >> 172.21.15.181:0/2346315750 conn(0x7fc4f406d610 msgr2=0x7fc4f406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/2346315750 shutdown_connections 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.771+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/2346315750 wait complete. 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.772+0000 7fc4fbeeb640 1 Processor -- start 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.772+0000 7fc4fbeeb640 1 -- start start 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f415a990 0x7fc4f4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415ad70 0x7fc4f40c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40d0c90 con 0x7fc4f40ce160 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40caa20 con 0x7fc4f415a990 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fbeeb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f40d0e10 con 0x7fc4f415ad70 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44906/0 (socket says 172.21.15.181:44906) 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 -- 172.21.15.181:0/1344024485 learned_addr learned my addr 172.21.15.181:0/1344024485 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4fa488640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415ad70 0x7fc4f40c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9486640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f415a990 0x7fc4f4154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 -- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415ad70 msgr2=0x7fc4f40c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415ad70 0x7fc4f40c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 -- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f415a990 msgr2=0x7fc4f4154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f415a990 0x7fc4f4154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 -- 172.21.15.181:0/1344024485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4153710 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4f9c87640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4f002e000 tx=0x7fc4f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.773+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f003b040 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.774+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f0046070 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.774+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f41539b0 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.774+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0041680 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.774+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f4154250 con 0x7fc4f40ce160 2024-03-29T04:10:05.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.774+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f4075900 con 0x7fc4f40ce160 2024-03-29T04:10:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.775+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc4f0041820 con 0x7fc4f40ce160 2024-03-29T04:10:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.775+0000 7fc4ea7fc640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 0x7fc4d806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.775+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4f00b02f0 con 0x7fc4f40ce160 2024-03-29T04:10:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.775+0000 7fc4f9486640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 0x7fc4d806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:05.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.776+0000 7fc4f9486640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 0x7fc4d806ea20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009560 tx=0x7fc4e4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:05.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.780+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc4f007e470 con 0x7fc4f40ce160 2024-03-29T04:10:05.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.965+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4f40728b0 con 0x7fc4f40ce160 2024-03-29T04:10:05.944 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.965+0000 7fc4ea7fc640 1 -- 172.21.15.181:0/1344024485 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fc4f007dd70 con 0x7fc4f40ce160 2024-03-29T04:10:05.944 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 msgr2=0x7fc4d806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 0x7fc4d806ea20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009560 tx=0x7fc4e4009190 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 msgr2=0x7fc4f4151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4f002e000 tx=0x7fc4f00046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 shutdown_connections 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc4d806c560 0x7fc4d806ea20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415ad70 0x7fc4f40c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f415a990 0x7fc4f4154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 --2- 172.21.15.181:0/1344024485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f40ce160 0x7fc4f4151e50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 >> 172.21.15.181:0/1344024485 conn(0x7fc4f406d610 msgr2=0x7fc4f4159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 shutdown_connections 2024-03-29T04:10:05.947 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:05.967+0000 7fc4fbeeb640 1 -- 172.21.15.181:0/1344024485 wait complete. 2024-03-29T04:10:05.957 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-29T04:10:05.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:05.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-29T04:10:05.985 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:10:05.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:05.986 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T04:10:06.041 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:06.041 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:06.058 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:06.058 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:06.117 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:06.117 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:06.169 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:06.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.290+0000 7efc37a7e640 1 Processor -- start 2024-03-29T04:10:06.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.290+0000 7efc37a7e640 1 -- start start 2024-03-29T04:10:06.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc301445e0 0x7efc300cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc30072340 0x7efc300cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc300727d0 0x7efc300cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7efc300cd610 con 0x7efc301445e0 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc301449c0 con 0x7efc30072340 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc37a7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc300cf5b0 con 0x7efc300727d0 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc301445e0 0x7efc300cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc301445e0 0x7efc300cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44914/0 (socket says 172.21.15.181:44914) 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 -- 172.21.15.181:0/3233127908 learned_addr learned my addr 172.21.15.181:0/3233127908 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc35019640 1 --2- 172.21.15.181:0/3233127908 >> v2:172.21.15.187:3300/0 conn(0x7efc30072340 0x7efc300cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3601b640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc300727d0 0x7efc300cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 -- 172.21.15.181:0/3233127908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc300727d0 msgr2=0x7efc300cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc300727d0 0x7efc300cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 -- 172.21.15.181:0/3233127908 >> v2:172.21.15.187:3300/0 conn(0x7efc30072340 msgr2=0x7efc300cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 --2- 172.21.15.181:0/3233127908 >> v2:172.21.15.187:3300/0 conn(0x7efc30072340 0x7efc300cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.291+0000 7efc3581a640 1 -- 172.21.15.181:0/3233127908 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc30144e10 con 0x7efc301445e0 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.292+0000 7efc3581a640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc301445e0 0x7efc300cf070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc300714b0 tx=0x7efc2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84ed6c01eb22456 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.292+0000 7efc34818640 1 -- 172.21.15.181:0/3233127908 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2802ca70 con 0x7efc301445e0 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.292+0000 7efc34818640 1 -- 172.21.15.181:0/3233127908 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2802cc10 con 0x7efc301445e0 2024-03-29T04:10:06.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.292+0000 7efc34818640 1 -- 172.21.15.181:0/3233127908 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280343f0 con 0x7efc301445e0 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 -- 172.21.15.181:0/3233127908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc301445e0 msgr2=0x7efc300cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc301445e0 0x7efc300cf070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc300714b0 tx=0x7efc2802bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 -- 172.21.15.181:0/3233127908 shutdown_connections 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc300727d0 0x7efc300cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 --2- 172.21.15.181:0/3233127908 >> v2:172.21.15.187:3300/0 conn(0x7efc30072340 0x7efc300cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 --2- 172.21.15.181:0/3233127908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc301445e0 0x7efc300cf070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 -- 172.21.15.181:0/3233127908 >> 172.21.15.181:0/3233127908 conn(0x7efc300b1c90 msgr2=0x7efc300b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 -- 172.21.15.181:0/3233127908 shutdown_connections 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.293+0000 7efc37a7e640 1 -- 172.21.15.181:0/3233127908 wait complete. 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.294+0000 7efc37a7e640 1 Processor -- start 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.294+0000 7efc37a7e640 1 -- start start 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc30072340 0x7efc301f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc301445e0 0x7efc300a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc300cad60 con 0x7efc300727d0 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc3015edb0 con 0x7efc301445e0 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc300cb040 con 0x7efc30072340 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44918/0 (socket says 172.21.15.181:44918) 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 -- 172.21.15.181:0/1143338752 learned_addr learned my addr 172.21.15.181:0/1143338752 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc3581a640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc30072340 0x7efc301f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc3601b640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc301445e0 0x7efc300a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 -- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc30072340 msgr2=0x7efc301f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc30072340 0x7efc301f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 -- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc301445e0 msgr2=0x7efc300a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc301445e0 0x7efc300a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 -- 172.21.15.181:0/1143338752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc300a99d0 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc35019640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e8c0 tx=0x7efc2000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20017070 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2000c940 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc300c72d0 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2000cd10 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.295+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc30132c80 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.296+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30075a10 con 0x7efc300727d0 2024-03-29T04:10:06.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.297+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc2001a070 con 0x7efc300727d0 2024-03-29T04:10:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.297+0000 7efc267fc640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 0x7efc1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.297+0000 7efc3581a640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 0x7efc1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.297+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efc2008b9e0 con 0x7efc300727d0 2024-03-29T04:10:06.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.298+0000 7efc3581a640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 0x7efc1406e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc28035000 tx=0x7efc28030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.302+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc200595b0 con 0x7efc300727d0 2024-03-29T04:10:06.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.478+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc300c78c0 con 0x7efc300727d0 2024-03-29T04:10:06.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.480+0000 7efc267fc640 1 -- 172.21.15.181:0/1143338752 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7efc2005d8b0 con 0x7efc300727d0 2024-03-29T04:10:06.459 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:06.460 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 msgr2=0x7efc1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 0x7efc1406e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc28035000 tx=0x7efc28030000 comp rx=0 tx=0).stop 2024-03-29T04:10:06.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 msgr2=0x7efc300b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e8c0 tx=0x7efc2000ed80 comp rx=0 tx=0).stop 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 shutdown_connections 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc301445e0 0x7efc300a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc300727d0 0x7efc300b15f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc1406c530 0x7efc1406e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 --2- 172.21.15.181:0/1143338752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc30072340 0x7efc301f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 >> 172.21.15.181:0/1143338752 conn(0x7efc300b1c90 msgr2=0x7efc300b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 shutdown_connections 2024-03-29T04:10:06.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.482+0000 7efc37a7e640 1 -- 172.21.15.181:0/1143338752 wait complete. 2024-03-29T04:10:06.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:06.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:06.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.597+0000 7f328bc62640 1 Processor -- start 2024-03-29T04:10:06.577 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.597+0000 7f328bc62640 1 -- start start 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328bc62640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f328406d8f0 0x7f328415fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328bc62640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32840cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328bc62640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32840ce9c0 con 0x7f328413f0c0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328bc62640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3284160e30 con 0x7f328406d8f0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328bc62640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3284160b50 con 0x7f32841603d0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f32899fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f328413f0c0 0x7f328406d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48732/0 (socket says 172.21.15.181:48732) 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f32899fe640 1 -- 172.21.15.181:0/3113370125 learned_addr learned my addr 172.21.15.181:0/3113370125 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328a1ff640 1 --2- 172.21.15.181:0/3113370125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32840cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f32891fd640 1 --2- 172.21.15.181:0/3113370125 >> v2:172.21.15.187:3300/0 conn(0x7f328406d8f0 0x7f328415fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f32889fc640 1 -- 172.21.15.181:0/3113370125 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741701795 0 0) 0x7f3280001110 con 0x7f328413f0c0 2024-03-29T04:10:06.579 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f32889fc640 1 -- 172.21.15.181:0/3113370125 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270003520 con 0x7f328413f0c0 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328a1ff640 1 -- 172.21.15.181:0/3113370125 >> v2:172.21.15.187:3300/0 conn(0x7f328406d8f0 msgr2=0x7f328415fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328a1ff640 1 --2- 172.21.15.181:0/3113370125 >> v2:172.21.15.187:3300/0 conn(0x7f328406d8f0 0x7f328415fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.598+0000 7f328a1ff640 1 -- 172.21.15.181:0/3113370125 >> v1:172.21.15.181:6789/0 conn(0x7f328413f0c0 legacy=0x7f328406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.599+0000 7f328a1ff640 1 -- 172.21.15.181:0/3113370125 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328413fa40 con 0x7f32841603d0 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.599+0000 7f328a1ff640 1 --2- 172.21.15.181:0/3113370125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32840cfbb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f327c009d90 tx=0x7f327c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0d38ffb600fb98 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.599+0000 7f32889fc640 1 -- 172.21.15.181:0/3113370125 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c039070 con 0x7f32841603d0 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.599+0000 7f32889fc640 1 -- 172.21.15.181:0/3113370125 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c02cb30 con 0x7f32841603d0 2024-03-29T04:10:06.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.599+0000 7f32889fc640 1 -- 172.21.15.181:0/3113370125 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c034440 con 0x7f32841603d0 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 -- 172.21.15.181:0/3113370125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 msgr2=0x7f32840cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 --2- 172.21.15.181:0/3113370125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32840cfbb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f327c009d90 tx=0x7f327c02bd80 comp rx=0 tx=0).stop 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 -- 172.21.15.181:0/3113370125 shutdown_connections 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 --2- 172.21.15.181:0/3113370125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32840cfbb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 --2- 172.21.15.181:0/3113370125 >> v2:172.21.15.187:3300/0 conn(0x7f328406d8f0 0x7f328415fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 -- 172.21.15.181:0/3113370125 >> 172.21.15.181:0/3113370125 conn(0x7f32840b5b00 msgr2=0x7f328406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 -- 172.21.15.181:0/3113370125 shutdown_connections 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.600+0000 7f328bc62640 1 -- 172.21.15.181:0/3113370125 wait complete. 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.601+0000 7f328bc62640 1 Processor -- start 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.601+0000 7f328bc62640 1 -- start start 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f328413f0c0 0x7f3284155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32841e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3284160840 con 0x7f328406d8f0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3284140000 con 0x7f328413f0c0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328bc62640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f328413f4a0 con 0x7f32841603d0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44922/0 (socket says 172.21.15.181:44922) 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 -- 172.21.15.181:0/2986279144 learned_addr learned my addr 172.21.15.181:0/2986279144 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32891fd640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f328413f0c0 0x7f3284155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f328a1ff640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32841e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 -- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 msgr2=0x7f32841e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32841e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 -- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f328413f0c0 msgr2=0x7f3284155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f328413f0c0 0x7f3284155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.602+0000 7f32899fe640 1 -- 172.21.15.181:0/2986279144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32841e6060 con 0x7f328406d8f0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f32899fe640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f328000c760 tx=0x7f328000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.584 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280017070 con 0x7f328406d8f0 2024-03-29T04:10:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3280005a70 con 0x7f328406d8f0 2024-03-29T04:10:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32841e6340 con 0x7f328406d8f0 2024-03-29T04:10:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280005e60 con 0x7f328406d8f0 2024-03-29T04:10:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.603+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841e6900 con 0x7f328406d8f0 2024-03-29T04:10:06.585 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.604+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284075890 con 0x7f328406d8f0 2024-03-29T04:10:06.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.604+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f328001a070 con 0x7f328406d8f0 2024-03-29T04:10:06.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.604+0000 7f327a7fc640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 0x7f326806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.605+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f328008ad40 con 0x7f328406d8f0 2024-03-29T04:10:06.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.605+0000 7f32891fd640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 0x7f326806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.586 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.605+0000 7f32891fd640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 0x7f326806e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32841f5b70 tx=0x7f3274009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.609+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f32840ce9c0 con 0x7f328406d8f0 2024-03-29T04:10:06.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.782+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32840cdcc0 con 0x7f328406d8f0 2024-03-29T04:10:06.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.782+0000 7f327a7fc640 1 -- 172.21.15.181:0/2986279144 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f3280058910 con 0x7f328406d8f0 2024-03-29T04:10:06.762 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 msgr2=0x7f326806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 0x7f326806e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32841f5b70 tx=0x7f3274009190 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 msgr2=0x7f32841e7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f328000c760 tx=0x7f328000cc20 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 shutdown_connections 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f326806c530 0x7f326806e9f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32841603d0 0x7f32841e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f328413f0c0 0x7f3284155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 --2- 172.21.15.181:0/2986279144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328406d8f0 0x7f32841e7100 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 >> 172.21.15.181:0/2986279144 conn(0x7f32840b5b00 msgr2=0x7f328415fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 shutdown_connections 2024-03-29T04:10:06.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.784+0000 7f328bc62640 1 -- 172.21.15.181:0/2986279144 wait complete. 2024-03-29T04:10:06.775 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.898+0000 7f19f0e55640 1 Processor -- start 2024-03-29T04:10:06.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.898+0000 7f19f0e55640 1 -- start start 2024-03-29T04:10:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec0b5dc0 0x7f19ec0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19ec0ba090 0x7f19ec0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 0x7f19ec06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0b66e0 con 0x7f19ec0ba090 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f19ec0d2ff0 con 0x7f19ec06c620 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19f0e55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec0d32d0 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19e9d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19ec0ba090 0x7f19ec0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19e9d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19ec0ba090 0x7f19ec0d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44934/0 (socket says 172.21.15.181:44934) 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19e9d9b640 1 -- 172.21.15.181:0/573021361 learned_addr learned my addr 172.21.15.181:0/573021361 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19ead9d640 1 --2- 172.21.15.181:0/573021361 >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 0x7f19ec06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.899+0000 7f19ea59c640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec0b5dc0 0x7f19ec0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 -- 172.21.15.181:0/573021361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec0b5dc0 msgr2=0x7f19ec0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec0b5dc0 0x7f19ec0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 -- 172.21.15.181:0/573021361 >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 msgr2=0x7f19ec06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:06.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/573021361 >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 0x7f19ec06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 -- 172.21.15.181:0/573021361 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec0d35b0 con 0x7f19ec0ba090 2024-03-29T04:10:06.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19ead9d640 1 --2- 172.21.15.181:0/573021361 >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 0x7f19ec06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.900+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0ba090 0x7f19ec0d2680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0d2120 tx=0x7f19e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211bd32a4f1a2d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19e959a640 1 -- 172.21.15.181:0/573021361 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0035020 con 0x7f19ec0ba090 2024-03-29T04:10:06.882 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19e959a640 1 -- 172.21.15.181:0/573021361 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0031870 con 0x7f19ec0ba090 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19e959a640 1 -- 172.21.15.181:0/573021361 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0031c40 con 0x7f19ec0ba090 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 -- 172.21.15.181:0/573021361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0ba090 msgr2=0x7f19ec0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0ba090 0x7f19ec0d2680 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0d2120 tx=0x7f19e002b910 comp rx=0 tx=0).stop 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 -- 172.21.15.181:0/573021361 shutdown_connections 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 --2- 172.21.15.181:0/573021361 >> v2:172.21.15.187:3300/0 conn(0x7f19ec06c620 0x7f19ec06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0ba090 0x7f19ec0d2680 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 --2- 172.21.15.181:0/573021361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec0b5dc0 0x7f19ec0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 -- 172.21.15.181:0/573021361 >> 172.21.15.181:0/573021361 conn(0x7f19ec0678f0 msgr2=0x7f19ec0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:06.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 -- 172.21.15.181:0/573021361 shutdown_connections 2024-03-29T04:10:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.901+0000 7f19f0e55640 1 -- 172.21.15.181:0/573021361 wait complete. 2024-03-29T04:10:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 Processor -- start 2024-03-29T04:10:06.884 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 -- start start 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 0x7f19ec0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19ec0ba090 0x7f19ec0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec0d4090 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec0d3b70 con 0x7f19ec0ba090 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.903+0000 7f19f0e55640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec15e880 con 0x7f19ec06c620 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44942/0 (socket says 172.21.15.181:44942) 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 -- 172.21.15.181:0/3207173918 learned_addr learned my addr 172.21.15.181:0/3207173918 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19ea59c640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 0x7f19ec0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.887 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19ead9d640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19ec0ba090 0x7f19ec0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 -- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 msgr2=0x7f19ec0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 0x7f19ec0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 -- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19ec0ba090 msgr2=0x7f19ec0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19ec0ba090 0x7f19ec0b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 -- 172.21.15.181:0/3207173918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec1f4a10 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19ea59c640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 0x7f19ec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19e9d9b640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f19e002b950 tx=0x7f19e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0040070 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0031e30 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec1f4cf0 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0035040 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.888 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.904+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec0cc7e0 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.905+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec075890 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.906+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f19e004b060 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.906+0000 7f19daffd640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 0x7f19cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.906+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f19e00ad5c0 con 0x7f19ec0b5dc0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.906+0000 7f19ea59c640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 0x7f19cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.907+0000 7f19ea59c640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 0x7f19cc06e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b4770 tx=0x7f19dc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:06.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:06.910+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f19e007b190 con 0x7f19ec0b5dc0 2024-03-29T04:10:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.086+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19ec0acd30 con 0x7f19ec0b5dc0 2024-03-29T04:10:07.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.086+0000 7f19daffd640 1 -- 172.21.15.181:0/3207173918 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f19e007f490 con 0x7f19ec0b5dc0 2024-03-29T04:10:07.066 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 msgr2=0x7f19cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 0x7f19cc06e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b4770 tx=0x7f19dc00b040 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 msgr2=0x7f19ec15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f19e002b950 tx=0x7f19e00046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 shutdown_connections 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19ec0ba090 0x7f19ec0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19ec0b5dc0 0x7f19ec15aa60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f19cc06c530 0x7f19cc06e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 --2- 172.21.15.181:0/3207173918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19ec06c620 0x7f19ec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 >> 172.21.15.181:0/3207173918 conn(0x7f19ec0678f0 msgr2=0x7f19ec160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 shutdown_connections 2024-03-29T04:10:07.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.088+0000 7f19f0e55640 1 -- 172.21.15.181:0/3207173918 wait complete. 2024-03-29T04:10:07.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.203+0000 7fdadad74640 1 Processor -- start 2024-03-29T04:10:07.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.203+0000 7fdadad74640 1 -- start start 2024-03-29T04:10:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdadad74640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdad40d4b40 0x7fdad40d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdadad74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdad40b9e30 0x7fdad40d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdadad74640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad40d2d00 con 0x7fdad415e840 2024-03-29T04:10:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdadad74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fdad40d2ad0 con 0x7fdad40b9e30 2024-03-29T04:10:07.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdadad74640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdad40d3910 con 0x7fdad40d4b40 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad8b10640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdad415e840 0x7fdad40d4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48748/0 (socket says 172.21.15.181:48748) 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad8b10640 1 -- 172.21.15.181:0/2170573730 learned_addr learned my addr 172.21.15.181:0/2170573730 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad9311640 1 --2- 172.21.15.181:0/2170573730 >> v2:172.21.15.181:3300/0 conn(0x7fdad40b9e30 0x7fdad40d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad3fff640 1 --2- 172.21.15.181:0/2170573730 >> v2:172.21.15.187:3300/0 conn(0x7fdad40d4b40 0x7fdad40d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad37fe640 1 -- 172.21.15.181:0/2170573730 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649056114 0 0) 0x7fdad40d2d00 con 0x7fdad415e840 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.204+0000 7fdad37fe640 1 -- 172.21.15.181:0/2170573730 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdabc003520 con 0x7fdad415e840 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad9311640 1 -- 172.21.15.181:0/2170573730 >> v2:172.21.15.187:3300/0 conn(0x7fdad40d4b40 msgr2=0x7fdad40d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad9311640 1 --2- 172.21.15.181:0/2170573730 >> v2:172.21.15.187:3300/0 conn(0x7fdad40d4b40 0x7fdad40d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad9311640 1 -- 172.21.15.181:0/2170573730 >> v1:172.21.15.181:6789/0 conn(0x7fdad415e840 legacy=0x7fdad40d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad9311640 1 -- 172.21.15.181:0/2170573730 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40d2fe0 con 0x7fdad40b9e30 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad9311640 1 --2- 172.21.15.181:0/2170573730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40b9e30 0x7fdad40d2590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdad40d4ff0 tx=0x7fdacc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1579ef6f9ea47e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.205+0000 7fdad37fe640 1 -- 172.21.15.181:0/2170573730 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc039070 con 0x7fdad40b9e30 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdad37fe640 1 -- 172.21.15.181:0/2170573730 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc02cbf0 con 0x7fdad40b9e30 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdad37fe640 1 -- 172.21.15.181:0/2170573730 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc034440 con 0x7fdad40b9e30 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 -- 172.21.15.181:0/2170573730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40b9e30 msgr2=0x7fdad40d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 --2- 172.21.15.181:0/2170573730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40b9e30 0x7fdad40d2590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdad40d4ff0 tx=0x7fdacc02be40 comp rx=0 tx=0).stop 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 -- 172.21.15.181:0/2170573730 shutdown_connections 2024-03-29T04:10:07.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 --2- 172.21.15.181:0/2170573730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40b9e30 0x7fdad40d2590 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 --2- 172.21.15.181:0/2170573730 >> v2:172.21.15.187:3300/0 conn(0x7fdad40d4b40 0x7fdad40d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 -- 172.21.15.181:0/2170573730 >> 172.21.15.181:0/2170573730 conn(0x7fdad40676e0 msgr2=0x7fdad4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 -- 172.21.15.181:0/2170573730 shutdown_connections 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.206+0000 7fdadad74640 1 -- 172.21.15.181:0/2170573730 wait complete. 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.207+0000 7fdadad74640 1 Processor -- start 2024-03-29T04:10:07.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.207+0000 7fdadad74640 1 -- start start 2024-03-29T04:10:07.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad40b9e30 0x7fdad40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 0x7fdad40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40b7640 con 0x7fdad40d4b40 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40d35a0 con 0x7fdad40b9e30 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdadad74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40b77c0 con 0x7fdad415e840 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44960/0 (socket says 172.21.15.181:44960) 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad3fff640 1 -- 172.21.15.181:0/2348304334 learned_addr learned my addr 172.21.15.181:0/2348304334 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad8b10640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad40b9e30 0x7fdad40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad9311640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 0x7fdad40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.208+0000 7fdad3fff640 1 -- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 msgr2=0x7fdad40cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad3fff640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 0x7fdad40cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad3fff640 1 -- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad40b9e30 msgr2=0x7fdad40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad3fff640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad40b9e30 0x7fdad40ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad3fff640 1 -- 172.21.15.181:0/2348304334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40cbb10 con 0x7fdad40d4b40 2024-03-29T04:10:07.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad9311640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 0x7fdad40cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad3fff640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e9b0 tx=0x7fdac800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8017070 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac800c9a0 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad40cbdf0 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac800cd90 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.209+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad40cc3b0 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.210+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4075890 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.210+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdac801a070 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.211+0000 7fdad17fa640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 0x7fdac006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.211+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdac808bd90 con 0x7fdad40d4b40 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.211+0000 7fdad8b10640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 0x7fdac006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.211+0000 7fdad8b10640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 0x7fdac006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac4005880 tx=0x7fdac4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.215+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdac80599f0 con 0x7fdad40d4b40 2024-03-29T04:10:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.388+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdad40cac80 con 0x7fdad40d4b40 2024-03-29T04:10:07.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.388+0000 7fdad17fa640 1 -- 172.21.15.181:0/2348304334 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fdac805dcf0 con 0x7fdad40d4b40 2024-03-29T04:10:07.368 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:10:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.390+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 msgr2=0x7fdac006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.390+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 0x7fdac006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac4005880 tx=0x7fdac4009040 comp rx=0 tx=0).stop 2024-03-29T04:10:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.390+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 msgr2=0x7fdad4154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.390+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e9b0 tx=0x7fdac800ee70 comp rx=0 tx=0).stop 2024-03-29T04:10:07.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 shutdown_connections 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fdac006c530 0x7fdac006e9f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdad415e840 0x7fdad40cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad40d4b40 0x7fdad4154010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 --2- 172.21.15.181:0/2348304334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad40b9e30 0x7fdad40ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 >> 172.21.15.181:0/2348304334 conn(0x7fdad40676e0 msgr2=0x7fdad415fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 shutdown_connections 2024-03-29T04:10:07.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.391+0000 7fdadad74640 1 -- 172.21.15.181:0/2348304334 wait complete. 2024-03-29T04:10:07.381 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.506+0000 7efdf85a8640 1 Processor -- start 2024-03-29T04:10:07.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.506+0000 7efdf85a8640 1 -- start start 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf85a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf85a8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf015fcb0 con 0x7efdf0144cb0 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf85a8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf00b35e0 con 0x7efdf015f800 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf85a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efdf00b3e80 con 0x7efdf0072d50 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf6344640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efdf0144cb0 0x7efdf0072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48750/0 (socket says 172.21.15.181:48750) 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf6344640 1 -- 172.21.15.181:0/3094202286 learned_addr learned my addr 172.21.15.181:0/3094202286 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5b43640 1 --2- 172.21.15.181:0/3094202286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599816547 0 0) 0x7efdec001110 con 0x7efdf0144cb0 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efddc003520 con 0x7efdf0144cb0 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761797957 0 0) 0x7efdf00b35e0 con 0x7efdf015f800 2024-03-29T04:10:07.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdec001110 con 0x7efdf015f800 2024-03-29T04:10:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5b43640 1 -- 172.21.15.181:0/3094202286 >> v1:172.21.15.187:6789/0 conn(0x7efdf015f800 legacy=0x7efdf00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5b43640 1 -- 172.21.15.181:0/3094202286 >> v1:172.21.15.181:6789/0 conn(0x7efdf0144cb0 legacy=0x7efdf0072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.507+0000 7efdf5b43640 1 -- 172.21.15.181:0/3094202286 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf00b38c0 con 0x7efdf0072d50 2024-03-29T04:10:07.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.508+0000 7efdf5b43640 1 --2- 172.21.15.181:0/3094202286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf00b2f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efde00097d0 tx=0x7efde002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1490ae597b0a891e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.508+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0035020 con 0x7efdf0072d50 2024-03-29T04:10:07.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.508+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde0031890 con 0x7efdf0072d50 2024-03-29T04:10:07.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.508+0000 7efdf5342640 1 -- 172.21.15.181:0/3094202286 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0031c60 con 0x7efdf0072d50 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 -- 172.21.15.181:0/3094202286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 msgr2=0x7efdf00b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3094202286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf00b2f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efde00097d0 tx=0x7efde002b930 comp rx=0 tx=0).stop 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 -- 172.21.15.181:0/3094202286 shutdown_connections 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3094202286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf00b2f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 -- 172.21.15.181:0/3094202286 >> 172.21.15.181:0/3094202286 conn(0x7efdf00b5b00 msgr2=0x7efdf006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 -- 172.21.15.181:0/3094202286 shutdown_connections 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.509+0000 7efdf85a8640 1 -- 172.21.15.181:0/3094202286 wait complete. 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.510+0000 7efdf85a8640 1 Processor -- start 2024-03-29T04:10:07.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.510+0000 7efdf85a8640 1 -- start start 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdf015f800 0x7efdf0149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efdf00ce760 con 0x7efdf0144cb0 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efdf00cdb30 con 0x7efdf015f800 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efdf00ce8e0 con 0x7efdf0072d50 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44972/0 (socket says 172.21.15.181:44972) 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 -- 172.21.15.181:0/3685095732 learned_addr learned my addr 172.21.15.181:0/3685095732 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf6344640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf6b45640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdf015f800 0x7efdf0149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 -- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 msgr2=0x7efdf014aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf014aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 -- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdf015f800 msgr2=0x7efdf0149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdf015f800 0x7efdf0149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 -- 172.21.15.181:0/3685095732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf0149e10 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf5b43640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efde00098f0 tx=0x7efde00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0035040 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde0030d20 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf014a0f0 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0006e10 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.511+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf014a6b0 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.513+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf0075890 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.513+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efde0038020 con 0x7efdf0144cb0 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.513+0000 7efde6ffd640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 0x7efdd406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.513+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efde00ae0e0 con 0x7efdf0144cb0 2024-03-29T04:10:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.513+0000 7efdf6344640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 0x7efdd406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.514+0000 7efdf6344640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 0x7efdd406e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdec000f60 tx=0x7efdec007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.518+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efde002d040 con 0x7efdf0144cb0 2024-03-29T04:10:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.692+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efdf00b4030 con 0x7efdf0144cb0 2024-03-29T04:10:07.671 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.692+0000 7efde6ffd640 1 -- 172.21.15.181:0/3685095732 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7efde007bc30 con 0x7efdf0144cb0 2024-03-29T04:10:07.671 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 msgr2=0x7efdd406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 0x7efdd406e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdec000f60 tx=0x7efdec007d70 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 msgr2=0x7efdf013d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efde00098f0 tx=0x7efde00046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 shutdown_connections 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdf015f800 0x7efdf0149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdf0144cb0 0x7efdf013d840 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efdd406c530 0x7efdd406e9f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 --2- 172.21.15.181:0/3685095732 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdf0072d50 0x7efdf014aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 >> 172.21.15.181:0/3685095732 conn(0x7efdf00b5b00 msgr2=0x7efdf006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 shutdown_connections 2024-03-29T04:10:07.674 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.694+0000 7efdf85a8640 1 -- 172.21.15.181:0/3685095732 wait complete. 2024-03-29T04:10:07.685 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:10:07.786 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.807+0000 7fb4236f1640 1 Processor -- start 2024-03-29T04:10:07.787 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.807+0000 7fb4236f1640 1 -- start start 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb4236f1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb41c071440 0x7fb41c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb4236f1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb41c0cca60 0x7fb41c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb4236f1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c071fe0 con 0x7fb41c15fba0 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb4236f1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c0cd3f0 con 0x7fb41c0cca60 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb4236f1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c071a20 con 0x7fb41c071440 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb421c8e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb41c0cca60 0x7fb41c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb42148d640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb41c15fba0 0x7fb41c160a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48078/0 (socket says 172.21.15.181:48078) 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb42148d640 1 -- 172.21.15.181:0/3054935453 learned_addr learned my addr 172.21.15.181:0/3054935453 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb420c8c640 1 --2- 172.21.15.181:0/3054935453 >> v2:172.21.15.187:3300/0 conn(0x7fb41c071440 0x7fb41c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb421c8e640 1 -- 172.21.15.181:0/3054935453 >> v2:172.21.15.187:3300/0 conn(0x7fb41c071440 msgr2=0x7fb41c0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:07.788 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb421c8e640 1 --2- 172.21.15.181:0/3054935453 >> v2:172.21.15.187:3300/0 conn(0x7fb41c071440 0x7fb41c0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb421c8e640 1 -- 172.21.15.181:0/3054935453 >> v1:172.21.15.181:6789/0 conn(0x7fb41c15fba0 legacy=0x7fb41c160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb421c8e640 1 -- 172.21.15.181:0/3054935453 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c071d00 con 0x7fb41c0cca60 2024-03-29T04:10:07.789 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.808+0000 7fb41bfff640 1 -- 172.21.15.181:0/3054935453 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452167679 0 0) 0x7fb41c071fe0 con 0x7fb41c15fba0 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.809+0000 7fb421c8e640 1 --2- 172.21.15.181:0/3054935453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c0cca60 0x7fb41c0cceb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0d0fa0 tx=0x7fb41402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e388bae53d2cf0b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.809+0000 7fb41bfff640 1 -- 172.21.15.181:0/3054935453 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414035020 con 0x7fb41c0cca60 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.809+0000 7fb41bfff640 1 -- 172.21.15.181:0/3054935453 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414031830 con 0x7fb41c0cca60 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.809+0000 7fb41bfff640 1 -- 172.21.15.181:0/3054935453 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414031c00 con 0x7fb41c0cca60 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 -- 172.21.15.181:0/3054935453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c0cca60 msgr2=0x7fb41c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3054935453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c0cca60 0x7fb41c0cceb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0d0fa0 tx=0x7fb41402b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 -- 172.21.15.181:0/3054935453 shutdown_connections 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3054935453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c0cca60 0x7fb41c0cceb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3054935453 >> v2:172.21.15.187:3300/0 conn(0x7fb41c071440 0x7fb41c0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 -- 172.21.15.181:0/3054935453 >> 172.21.15.181:0/3054935453 conn(0x7fb41c06d610 msgr2=0x7fb41c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.790 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 -- 172.21.15.181:0/3054935453 shutdown_connections 2024-03-29T04:10:07.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.810+0000 7fb4236f1640 1 -- 172.21.15.181:0/3054935453 wait complete. 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.811+0000 7fb4236f1640 1 Processor -- start 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.811+0000 7fb4236f1640 1 -- start start 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41c0cca60 0x7fb41c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c15fba0 0x7fb41c14b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0cb930 con 0x7fb41c071440 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0caed0 con 0x7fb41c15fba0 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb4236f1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c0b2f20 con 0x7fb41c0cca60 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33786/0 (socket says 172.21.15.181:33786) 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 -- 172.21.15.181:0/3635779088 learned_addr learned my addr 172.21.15.181:0/3635779088 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb420c8c640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41c0cca60 0x7fb41c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb421c8e640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c15fba0 0x7fb41c14b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 -- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41c0cca60 msgr2=0x7fb41c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41c0cca60 0x7fb41c0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 -- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c15fba0 msgr2=0x7fb41c14b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c15fba0 0x7fb41c14b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 -- 172.21.15.181:0/3635779088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c14e5d0 con 0x7fb41c071440 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.812+0000 7fb42148d640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c7c0 tx=0x7fb40c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.813+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c018070 con 0x7fb41c071440 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.813+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c00a5e0 con 0x7fb41c071440 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.813+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c14e2f0 con 0x7fb41c071440 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.813+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00a9d0 con 0x7fb41c071440 2024-03-29T04:10:07.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.813+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c0ae2f0 con 0x7fb41c071440 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.814+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c075900 con 0x7fb41c071440 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.814+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb40c01b070 con 0x7fb41c071440 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.814+0000 7fb419ffb640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 0x7fb40806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.814+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb40c08afe0 con 0x7fb41c071440 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.814+0000 7fb420c8c640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 0x7fb40806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:07.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.815+0000 7fb420c8c640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 0x7fb40806e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0abdb0 tx=0x7fb410009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:07.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:07.819+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb40c058bb0 con 0x7fb41c071440 2024-03-29T04:10:07.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.003+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb41c073050 con 0x7fb41c071440 2024-03-29T04:10:07.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.004+0000 7fb419ffb640 1 -- 172.21.15.181:0/3635779088 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fb40c05ceb0 con 0x7fb41c071440 2024-03-29T04:10:07.983 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:10:07.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 msgr2=0x7fb40806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 0x7fb40806e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0abdb0 tx=0x7fb410009190 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 msgr2=0x7fb41c14ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c7c0 tx=0x7fb40c00cc80 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 shutdown_connections 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c15fba0 0x7fb41c14b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb40806c530 0x7fb40806e9f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb41c0cca60 0x7fb41c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 --2- 172.21.15.181:0/3635779088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41c071440 0x7fb41c14ae20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 >> 172.21.15.181:0/3635779088 conn(0x7fb41c06d610 msgr2=0x7fb41c0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 shutdown_connections 2024-03-29T04:10:07.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.006+0000 7fb4236f1640 1 -- 172.21.15.181:0/3635779088 wait complete. 2024-03-29T04:10:07.996 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-29T04:10:07.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:07.996 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-29T04:10:08.028 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:10:08.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:08.028 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T04:10:08.084 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T04:10:08.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:08.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.206+0000 7f2483940640 1 Processor -- start 2024-03-29T04:10:08.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.206+0000 7f2483940640 1 -- start start 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2483940640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f247c15b270 0x7f247c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2483940640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f247c072280 0x7f247c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2483940640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247c15b650 con 0x7f247c0ccf10 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2483940640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f247c0cd380 con 0x7f247c072280 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2483940640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f247c072ab0 con 0x7f247c15b270 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2480edb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f247c072280 0x7f247c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2481edd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f247c0ccf10 0x7f247c0cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48088/0 (socket says 172.21.15.181:48088) 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2481edd640 1 -- 172.21.15.181:0/3190111593 learned_addr learned my addr 172.21.15.181:0/3190111593 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f24816dc640 1 --2- 172.21.15.181:0/3190111593 >> v2:172.21.15.187:3300/0 conn(0x7f247c15b270 0x7f247c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2480edb640 1 -- 172.21.15.181:0/3190111593 >> v2:172.21.15.187:3300/0 conn(0x7f247c15b270 msgr2=0x7f247c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:08.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2480edb640 1 --2- 172.21.15.181:0/3190111593 >> v2:172.21.15.187:3300/0 conn(0x7f247c15b270 0x7f247c15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2480edb640 1 -- 172.21.15.181:0/3190111593 >> v1:172.21.15.181:6789/0 conn(0x7f247c0ccf10 legacy=0x7f247c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2473fff640 1 -- 172.21.15.181:0/3190111593 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363042688 0 0) 0x7f247c15b650 con 0x7f247c0ccf10 2024-03-29T04:10:08.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.207+0000 7f2480edb640 1 -- 172.21.15.181:0/3190111593 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c072d90 con 0x7f247c072280 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.208+0000 7f2480edb640 1 --2- 172.21.15.181:0/3190111593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c072280 0x7f247c0cc9d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f247c06fa10 tx=0x7f246c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a769cf1d8994aff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.208+0000 7f2473fff640 1 -- 172.21.15.181:0/3190111593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c035020 con 0x7f247c072280 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.208+0000 7f2473fff640 1 -- 172.21.15.181:0/3190111593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c031830 con 0x7f247c072280 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.208+0000 7f2473fff640 1 -- 172.21.15.181:0/3190111593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c031c00 con 0x7f247c072280 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 -- 172.21.15.181:0/3190111593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c072280 msgr2=0x7f247c0cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 --2- 172.21.15.181:0/3190111593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c072280 0x7f247c0cc9d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f247c06fa10 tx=0x7f246c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 -- 172.21.15.181:0/3190111593 shutdown_connections 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 --2- 172.21.15.181:0/3190111593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c072280 0x7f247c0cc9d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 --2- 172.21.15.181:0/3190111593 >> v2:172.21.15.187:3300/0 conn(0x7f247c15b270 0x7f247c15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 -- 172.21.15.181:0/3190111593 >> 172.21.15.181:0/3190111593 conn(0x7f247c0b1c90 msgr2=0x7f247c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 -- 172.21.15.181:0/3190111593 shutdown_connections 2024-03-29T04:10:08.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.209+0000 7f2483940640 1 -- 172.21.15.181:0/3190111593 wait complete. 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.210+0000 7f2483940640 1 Processor -- start 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.210+0000 7f2483940640 1 -- start start 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f247c072280 0x7f247c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f247c0ccf10 0x7f247c15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f247c073630 con 0x7f247c15b270 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f247c073350 con 0x7f247c0ccf10 2024-03-29T04:10:08.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2483940640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f247c0726e0 con 0x7f247c072280 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33802/0 (socket says 172.21.15.181:33802) 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 -- 172.21.15.181:0/1573667166 learned_addr learned my addr 172.21.15.181:0/1573667166 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f24816dc640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f247c072280 0x7f247c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2480edb640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f247c0ccf10 0x7f247c15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 -- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f247c072280 msgr2=0x7f247c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f247c072280 0x7f247c153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 -- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f247c0ccf10 msgr2=0x7f247c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f247c0ccf10 0x7f247c15cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 -- 172.21.15.181:0/1573667166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c15d970 con 0x7f247c15b270 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2481edd640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f247400c450 tx=0x7f247400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.211+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474018070 con 0x7f247c15b270 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.212+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474007500 con 0x7f247c15b270 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.212+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c1f42c0 con 0x7f247c15b270 2024-03-29T04:10:08.193 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.212+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474004a20 con 0x7f247c15b270 2024-03-29T04:10:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.212+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c15e550 con 0x7f247c15b270 2024-03-29T04:10:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.213+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c075a10 con 0x7f247c15b270 2024-03-29T04:10:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.213+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f247401b070 con 0x7f247c15b270 2024-03-29T04:10:08.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.213+0000 7f2471ffb640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.214+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f247408ac50 con 0x7f247c15b270 2024-03-29T04:10:08.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.214+0000 7f24816dc640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.214+0000 7f24816dc640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 0x7f246006e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2478011340 tx=0x7f247800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.197 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.218+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2474090020 con 0x7f247c15b270 2024-03-29T04:10:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.431+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247c0cd720 con 0x7f247c15b270 2024-03-29T04:10:08.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.432+0000 7f2471ffb640 1 -- 172.21.15.181:0/1573667166 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f24740588b0 con 0x7f247c15b270 2024-03-29T04:10:08.412 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:08.412 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:10:08.413 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 msgr2=0x7f246006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 0x7f246006e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2478011340 tx=0x7f247800b040 comp rx=0 tx=0).stop 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 msgr2=0x7f247c14c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f247400c450 tx=0x7f247400c910 comp rx=0 tx=0).stop 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 shutdown_connections 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f247c15b270 0x7f247c14c710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f247c0ccf10 0x7f247c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f246006c530 0x7f246006e9f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 --2- 172.21.15.181:0/1573667166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f247c072280 0x7f247c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 >> 172.21.15.181:0/1573667166 conn(0x7f247c0b1c90 msgr2=0x7f247c0b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 shutdown_connections 2024-03-29T04:10:08.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.434+0000 7f2483940640 1 -- 172.21.15.181:0/1573667166 wait complete. 2024-03-29T04:10:08.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.547+0000 7f3c7ad22640 1 Processor -- start 2024-03-29T04:10:08.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.547+0000 7f3c7ad22640 1 -- start start 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3c7415e950 0x7f3c740d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c740d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c740d2f00 0x7f3c740cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f3c740d3cb0 con 0x7f3c740d2f00 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3c740d4d00 con 0x7f3c7415e950 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c7ad22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c740d36f0 con 0x7f3c740d48c0 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c792bf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c740d2f00 0x7f3c740cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c792bf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c740d2f00 0x7f3c740cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33816/0 (socket says 172.21.15.181:33816) 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c792bf640 1 -- 172.21.15.181:0/2119380079 learned_addr learned my addr 172.21.15.181:0/2119380079 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c78abe640 1 --2- 172.21.15.181:0/2119380079 >> v2:172.21.15.187:3300/0 conn(0x7f3c7415e950 0x7f3c740d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.548+0000 7f3c73fff640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c740d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 -- 172.21.15.181:0/2119380079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 msgr2=0x7f3c740d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c740d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 -- 172.21.15.181:0/2119380079 >> v2:172.21.15.187:3300/0 conn(0x7f3c7415e950 msgr2=0x7f3c740d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:08.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 --2- 172.21.15.181:0/2119380079 >> v2:172.21.15.187:3300/0 conn(0x7f3c7415e950 0x7f3c740d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 -- 172.21.15.181:0/2119380079 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c740d39d0 con 0x7f3c740d2f00 2024-03-29T04:10:08.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c792bf640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c740d2f00 0x7f3c740cef90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3c740d5510 tx=0x7f3c6c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a977d1d9cd07b064 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c737fe640 1 -- 172.21.15.181:0/2119380079 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c02cb00 con 0x7f3c740d2f00 2024-03-29T04:10:08.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c737fe640 1 -- 172.21.15.181:0/2119380079 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c02cca0 con 0x7f3c740d2f00 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.549+0000 7f3c737fe640 1 -- 172.21.15.181:0/2119380079 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c034610 con 0x7f3c740d2f00 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 -- 172.21.15.181:0/2119380079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c740d2f00 msgr2=0x7f3c740cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c740d2f00 0x7f3c740cef90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3c740d5510 tx=0x7f3c6c02bf60 comp rx=0 tx=0).stop 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 -- 172.21.15.181:0/2119380079 shutdown_connections 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c740d2f00 0x7f3c740cef90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/2119380079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c740d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/2119380079 >> v2:172.21.15.187:3300/0 conn(0x7f3c7415e950 0x7f3c740d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 -- 172.21.15.181:0/2119380079 >> 172.21.15.181:0/2119380079 conn(0x7f3c740676e0 msgr2=0x7f3c74067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 -- 172.21.15.181:0/2119380079 shutdown_connections 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.550+0000 7f3c7ad22640 1 -- 172.21.15.181:0/2119380079 wait complete. 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.551+0000 7f3c7ad22640 1 Processor -- start 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.551+0000 7f3c7ad22640 1 -- start start 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 0x7f3c7414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c741600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740d3390 con 0x7f3c7415e950 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740b7da0 con 0x7f3c740d2f00 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c7ad22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7415ed30 con 0x7f3c740d48c0 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33822/0 (socket says 172.21.15.181:33822) 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 -- 172.21.15.181:0/1912284267 learned_addr learned my addr 172.21.15.181:0/1912284267 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c78abe640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 0x7f3c7414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c73fff640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c741600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 -- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 msgr2=0x7f3c741600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c741600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 -- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 msgr2=0x7f3c7414a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 0x7f3c7414a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c792bf640 1 -- 172.21.15.181:0/1912284267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c740ca7a0 con 0x7f3c7415e950 2024-03-29T04:10:08.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.552+0000 7f3c78abe640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 0x7f3c7414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c792bf640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f40 tx=0x7f3c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c043020 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c02ce50 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c740caa50 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c009510 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.553+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7414af50 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.554+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c74075a10 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.554+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3c6c046020 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.555+0000 7f3c717fa640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.555+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3c6c0adee0 con 0x7f3c7415e950 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.555+0000 7f3c78abe640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.555+0000 7f3c78abe640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 0x7f3c6006e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009eb0 tx=0x7f3c64009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.559+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3c6c04d030 con 0x7f3c7415e950 2024-03-29T04:10:08.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.737+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c740cdcb0 con 0x7f3c7415e950 2024-03-29T04:10:08.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.738+0000 7f3c717fa640 1 -- 172.21.15.181:0/1912284267 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7f3c6c07bab0 con 0x7f3c7415e950 2024-03-29T04:10:08.718 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:08.718 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 msgr2=0x7f3c6006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 0x7f3c6006e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009eb0 tx=0x7f3c64009190 comp rx=0 tx=0).stop 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 msgr2=0x7f3c741605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009f40 tx=0x7f3c6c004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 shutdown_connections 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c7415e950 0x7f3c741605f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c740d48c0 0x7f3c741600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 --2- 172.21.15.181:0/1912284267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c740d2f00 0x7f3c7414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 >> 172.21.15.181:0/1912284267 conn(0x7f3c740676e0 msgr2=0x7f3c740cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 shutdown_connections 2024-03-29T04:10:08.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.740+0000 7f3c7ad22640 1 -- 172.21.15.181:0/1912284267 wait complete. 2024-03-29T04:10:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:09:13.613655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:09:14.603835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T04:09:18.323786+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T04:10:08.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.855+0000 7faf0e13d640 1 Processor -- start 2024-03-29T04:10:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.855+0000 7faf0e13d640 1 -- start start 2024-03-29T04:10:08.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf0e13d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf080cf410 0x7faf0806c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf0e13d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf081476b0 con 0x7faf081472d0 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf0e13d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf08071290 con 0x7faf0806db00 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf0e13d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7faf08148140 con 0x7faf080cf410 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf07fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf080cf410 0x7faf0806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf077fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faf081472d0 0x7faf0806d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48094/0 (socket says 172.21.15.181:48094) 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.856+0000 7faf077fe640 1 -- 172.21.15.181:0/1910311825 learned_addr learned my addr 172.21.15.181:0/1910311825 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.857+0000 7faf07fff640 1 -- 172.21.15.181:0/1910311825 >> v1:172.21.15.187:6789/0 conn(0x7faf0806db00 legacy=0x7faf08162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.857+0000 7faf07fff640 1 -- 172.21.15.181:0/1910311825 >> v1:172.21.15.181:6789/0 conn(0x7faf081472d0 legacy=0x7faf0806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.857+0000 7faf067fc640 1 -- 172.21.15.181:0/1910311825 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461873773 0 0) 0x7faf08071290 con 0x7faf0806db00 2024-03-29T04:10:08.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.857+0000 7faf07fff640 1 -- 172.21.15.181:0/1910311825 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf08071570 con 0x7faf080cf410 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.857+0000 7faf07fff640 1 --2- 172.21.15.181:0/1910311825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf080cf410 0x7faf0806c690 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf00009d90 tx=0x7faf0002bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0cd71c99ca61c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf067fc640 1 -- 172.21.15.181:0/1910311825 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00039070 con 0x7faf080cf410 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf067fc640 1 -- 172.21.15.181:0/1910311825 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0002ca20 con 0x7faf080cf410 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf067fc640 1 -- 172.21.15.181:0/1910311825 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0002cdf0 con 0x7faf080cf410 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 -- 172.21.15.181:0/1910311825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf080cf410 msgr2=0x7faf0806c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 --2- 172.21.15.181:0/1910311825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf080cf410 0x7faf0806c690 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf00009d90 tx=0x7faf0002bcf0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 -- 172.21.15.181:0/1910311825 shutdown_connections 2024-03-29T04:10:08.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 --2- 172.21.15.181:0/1910311825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf080cf410 0x7faf0806c690 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 -- 172.21.15.181:0/1910311825 >> 172.21.15.181:0/1910311825 conn(0x7faf080b3e70 msgr2=0x7faf080b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:08.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 -- 172.21.15.181:0/1910311825 shutdown_connections 2024-03-29T04:10:08.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.858+0000 7faf0e13d640 1 -- 172.21.15.181:0/1910311825 wait complete. 2024-03-29T04:10:08.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.859+0000 7faf0e13d640 1 Processor -- start 2024-03-29T04:10:08.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.859+0000 7faf0e13d640 1 -- start start 2024-03-29T04:10:08.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf080cf410 0x7faf081e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf081472d0 0x7faf081f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf08071b30 con 0x7faf0806db00 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf08071040 con 0x7faf080cf410 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf0e13d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf080d0940 con 0x7faf081472d0 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33840/0 (socket says 172.21.15.181:33840) 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 -- 172.21.15.181:0/3100177023 learned_addr learned my addr 172.21.15.181:0/3100177023 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf07fff640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf081472d0 0x7faf081f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf06ffd640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf080cf410 0x7faf081e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 -- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf081472d0 msgr2=0x7faf081f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf081472d0 0x7faf081f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 -- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf080cf410 msgr2=0x7faf081e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf080cf410 0x7faf081e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.860+0000 7faf077fe640 1 -- 172.21.15.181:0/3100177023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081e4830 con 0x7faf0806db00 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faf077fe640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faef800c520 tx=0x7faef800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8017070 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80074e0 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf08142e40 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8004a00 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.861+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf08143400 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.862+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf08075890 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.862+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faef801a070 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.863+0000 7faeeffff640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 0x7faef406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.863+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faef808ae80 con 0x7faf0806db00 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.863+0000 7faf06ffd640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 0x7faef406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:08.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.863+0000 7faf06ffd640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 0x7faef406e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf0806cb90 tx=0x7faefc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:08.846 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:08.867+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faef8058a60 con 0x7faf0806db00 2024-03-29T04:10:09.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.044+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf0806c8e0 con 0x7faf0806db00 2024-03-29T04:10:09.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.045+0000 7faeeffff640 1 -- 172.21.15.181:0/3100177023 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7faef805cd60 con 0x7faf0806db00 2024-03-29T04:10:09.024 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:09.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 msgr2=0x7faef406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 0x7faef406e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faf0806cb90 tx=0x7faefc009040 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 msgr2=0x7faf081e38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faef800c520 tx=0x7faef800c9e0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 shutdown_connections 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faef406c530 0x7faef406e9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf081472d0 0x7faf081f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf080cf410 0x7faf081e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 --2- 172.21.15.181:0/3100177023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf0806db00 0x7faf081e38e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 >> 172.21.15.181:0/3100177023 conn(0x7faf080b3e70 msgr2=0x7faf080d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 shutdown_connections 2024-03-29T04:10:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.047+0000 7faf0e13d640 1 -- 172.21.15.181:0/3100177023 wait complete. 2024-03-29T04:10:09.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:10:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.160+0000 7faf58f60640 1 Processor -- start 2024-03-29T04:10:09.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.160+0000 7faf58f60640 1 -- start start 2024-03-29T04:10:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.161+0000 7faf58f60640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf54155b10 0x7faf540b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.161+0000 7faf58f60640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faf540d03c0 0x7faf540d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.161+0000 7faf58f60640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf540b5ff0 con 0x7faf5415fae0 2024-03-29T04:10:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.161+0000 7faf58f60640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7faf54155ef0 con 0x7faf54155b10 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.161+0000 7faf58f60640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faf540b41b0 con 0x7faf540d03c0 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf54155b10 0x7faf540b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faf54155b10 0x7faf540b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33848/0 (socket says 172.21.15.181:33848) 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 -- 172.21.15.181:0/1114397629 learned_addr learned my addr 172.21.15.181:0/1114397629 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf52d9d640 1 --2- 172.21.15.181:0/1114397629 >> v2:172.21.15.187:3300/0 conn(0x7faf540d03c0 0x7faf540d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 -- 172.21.15.181:0/1114397629 >> v2:172.21.15.187:3300/0 conn(0x7faf540d03c0 msgr2=0x7faf540d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 --2- 172.21.15.181:0/1114397629 >> v2:172.21.15.187:3300/0 conn(0x7faf540d03c0 0x7faf540d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 -- 172.21.15.181:0/1114397629 >> v1:172.21.15.187:6789/0 conn(0x7faf5415fae0 legacy=0x7faf540b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.162+0000 7faf5259c640 1 -- 172.21.15.181:0/1114397629 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf540b4490 con 0x7faf54155b10 2024-03-29T04:10:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf5259c640 1 --2- 172.21.15.181:0/1114397629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf540b4c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf540b56d0 tx=0x7faf4402e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f317545fb4ef4f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf5159a640 1 -- 172.21.15.181:0/1114397629 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400b0d0 con 0x7faf54155b10 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf5159a640 1 -- 172.21.15.181:0/1114397629 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4400b270 con 0x7faf54155b10 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf5159a640 1 -- 172.21.15.181:0/1114397629 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44009600 con 0x7faf54155b10 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 -- 172.21.15.181:0/1114397629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 msgr2=0x7faf540b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 --2- 172.21.15.181:0/1114397629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf540b4c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf540b56d0 tx=0x7faf4402e180 comp rx=0 tx=0).stop 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 -- 172.21.15.181:0/1114397629 shutdown_connections 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 --2- 172.21.15.181:0/1114397629 >> v2:172.21.15.187:3300/0 conn(0x7faf540d03c0 0x7faf540d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 --2- 172.21.15.181:0/1114397629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf540b4c90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 -- 172.21.15.181:0/1114397629 >> 172.21.15.181:0/1114397629 conn(0x7faf540d1160 msgr2=0x7faf540d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 -- 172.21.15.181:0/1114397629 shutdown_connections 2024-03-29T04:10:09.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.163+0000 7faf58f60640 1 -- 172.21.15.181:0/1114397629 wait complete. 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 Processor -- start 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 -- start start 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf540d03c0 0x7faf54142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 0x7faf540ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.165+0000 7faf58f60640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf5415b9d0 con 0x7faf54155b10 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf58f60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf540b62d0 con 0x7faf5415fae0 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf58f60640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf5415bb50 con 0x7faf540d03c0 2024-03-29T04:10:09.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33858/0 (socket says 172.21.15.181:33858) 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 -- 172.21.15.181:0/4204500000 learned_addr learned my addr 172.21.15.181:0/4204500000 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf5259c640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf540d03c0 0x7faf54142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf52d9d640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 0x7faf540ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 -- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf540d03c0 msgr2=0x7faf54142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf540d03c0 0x7faf54142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 -- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 msgr2=0x7faf540ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 0x7faf540ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 -- 172.21.15.181:0/4204500000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf54150270 con 0x7faf54155b10 2024-03-29T04:10:09.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf52d9d640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 0x7faf540ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf51d9b640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c3c0 tx=0x7faf4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480052d0 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf48005490 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf54150550 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48007260 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.166+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf54150b10 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.167+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf540b0930 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.168+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faf48007400 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.168+0000 7faf42ffd640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.168+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faf4808af30 con 0x7faf54155b10 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.168+0000 7faf5259c640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.168+0000 7faf5259c640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 0x7faf3406e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf44037000 tx=0x7faf44032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.172+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faf540b0930 con 0x7faf54155b10 2024-03-29T04:10:09.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.345+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf54075890 con 0x7faf54155b10 2024-03-29T04:10:09.325 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.345+0000 7faf42ffd640 1 -- 172.21.15.181:0/4204500000 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7faf48058b00 con 0x7faf54155b10 2024-03-29T04:10:09.325 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:09.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 msgr2=0x7faf3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 0x7faf3406e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf44037000 tx=0x7faf44032000 comp rx=0 tx=0).stop 2024-03-29T04:10:09.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 msgr2=0x7faf5414f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c3c0 tx=0x7faf4800c880 comp rx=0 tx=0).stop 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 shutdown_connections 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf5415fae0 0x7faf540ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf54155b10 0x7faf5414f790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 --2- 172.21.15.181:0/4204500000 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf540d03c0 0x7faf54142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 >> 172.21.15.181:0/4204500000 conn(0x7faf540d1160 msgr2=0x7faf5406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 shutdown_connections 2024-03-29T04:10:09.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.347+0000 7faf58f60640 1 -- 172.21.15.181:0/4204500000 wait complete. 2024-03-29T04:10:09.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:10:09.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.463+0000 7f59d584a640 1 Processor -- start 2024-03-29T04:10:09.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.463+0000 7f59d584a640 1 -- start start 2024-03-29T04:10:09.445 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59d584a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59d0072d50 0x7f59d00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59d584a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d015fc70 con 0x7f59d015f800 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59d584a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d00b35e0 con 0x7f59d0144cb0 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59d584a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00b3e80 con 0x7f59d0072d50 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59ceffd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f59d0144cb0 0x7f59d0072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:56866/0 (socket says 172.21.15.181:56866) 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59ceffd640 1 -- 172.21.15.181:0/3537775378 learned_addr learned my addr 172.21.15.181:0/3537775378 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59cf7fe640 1 --1- 172.21.15.181:0/3537775378 >> v1:172.21.15.181:6789/0 conn(0x7f59d015f800 0x7f59d00ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48098/0 (socket says 172.21.15.181:48098) 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59c67fc640 1 --2- 172.21.15.181:0/3537775378 >> v2:172.21.15.181:3300/0 conn(0x7f59d0072d50 0x7f59d00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709785405 0 0) 0x7f59d00b35e0 con 0x7f59d0144cb0 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59bc003520 con 0x7f59d0144cb0 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59c67fc640 1 -- 172.21.15.181:0/3537775378 >> v1:172.21.15.187:6789/0 conn(0x7f59d0144cb0 legacy=0x7f59d0072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59c67fc640 1 -- 172.21.15.181:0/3537775378 >> v1:172.21.15.181:6789/0 conn(0x7f59d015f800 legacy=0x7f59d00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356200831 0 0) 0x7f59d015fc70 con 0x7f59d015f800 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.464+0000 7f59c67fc640 1 -- 172.21.15.181:0/3537775378 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00b38c0 con 0x7f59d0072d50 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.465+0000 7f59c67fc640 1 --2- 172.21.15.181:0/3537775378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0072d50 0x7f59d00b2f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59d00b4000 tx=0x7f59b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c932dc813250e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.465+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8034070 con 0x7f59d0072d50 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.465+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8030610 con 0x7f59d0072d50 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.465+0000 7f59ce7fc640 1 -- 172.21.15.181:0/3537775378 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80309c0 con 0x7f59d0072d50 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 -- 172.21.15.181:0/3537775378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0072d50 msgr2=0x7f59d00b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 --2- 172.21.15.181:0/3537775378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0072d50 0x7f59d00b2f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f59d00b4000 tx=0x7f59b802b960 comp rx=0 tx=0).stop 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 -- 172.21.15.181:0/3537775378 shutdown_connections 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 --2- 172.21.15.181:0/3537775378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0072d50 0x7f59d00b2f20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 -- 172.21.15.181:0/3537775378 >> 172.21.15.181:0/3537775378 conn(0x7f59d00b5b00 msgr2=0x7f59d006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 -- 172.21.15.181:0/3537775378 shutdown_connections 2024-03-29T04:10:09.447 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.466+0000 7f59d584a640 1 -- 172.21.15.181:0/3537775378 wait complete. 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.467+0000 7f59d584a640 1 Processor -- start 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.467+0000 7f59d584a640 1 -- start start 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d0072d50 0x7f59d01f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d015f800 0x7f59d00aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.449 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00ce760 con 0x7f59d0144cb0 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00cdb30 con 0x7f59d015f800 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59d584a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00ce8e0 con 0x7f59d0072d50 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33874/0 (socket says 172.21.15.181:33874) 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 -- 172.21.15.181:0/172935786 learned_addr learned my addr 172.21.15.181:0/172935786 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59ceffd640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d0072d50 0x7f59d01f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59cf7fe640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d015f800 0x7f59d00aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 -- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d0072d50 msgr2=0x7f59d01f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d0072d50 0x7f59d01f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 -- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d015f800 msgr2=0x7f59d00aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d015f800 0x7f59d00aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.468+0000 7f59c67fc640 1 -- 172.21.15.181:0/172935786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d01f2d60 con 0x7f59d0144cb0 2024-03-29T04:10:09.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.469+0000 7f59c67fc640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f59b8000c00 tx=0x7f59b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.472+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8034040 con 0x7f59d0144cb0 2024-03-29T04:10:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.472+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8030410 con 0x7f59d0144cb0 2024-03-29T04:10:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.472+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8039c50 con 0x7f59d0144cb0 2024-03-29T04:10:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.472+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d015b8f0 con 0x7f59d0144cb0 2024-03-29T04:10:09.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.472+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d015beb0 con 0x7f59d0144cb0 2024-03-29T04:10:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.473+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00d0bf0 con 0x7f59d0144cb0 2024-03-29T04:10:09.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.473+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f59b8037070 con 0x7f59d0144cb0 2024-03-29T04:10:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.474+0000 7f59ccff9640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 0x7f59b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.474+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f59b80ad9a0 con 0x7f59d0144cb0 2024-03-29T04:10:09.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.474+0000 7f59ceffd640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 0x7f59b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.474+0000 7f59ceffd640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 0x7f59b006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59c00011e0 tx=0x7f59c0007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.478+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f59d00d0bf0 con 0x7f59d0144cb0 2024-03-29T04:10:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.653+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59d00ccef0 con 0x7f59d0144cb0 2024-03-29T04:10:09.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.654+0000 7f59ccff9640 1 -- 172.21.15.181:0/172935786 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f59d00ccef0 con 0x7f59d0144cb0 2024-03-29T04:10:09.634 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 msgr2=0x7f59b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 0x7f59b006e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59c00011e0 tx=0x7f59c0007d30 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 msgr2=0x7f59d015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f59b8000c00 tx=0x7f59b80046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 shutdown_connections 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59d015f800 0x7f59d00aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59d0144cb0 0x7f59d015ab40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f59b006c530 0x7f59b006e9f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 --2- 172.21.15.181:0/172935786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59d0072d50 0x7f59d01f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 >> 172.21.15.181:0/172935786 conn(0x7f59d00b5b00 msgr2=0x7f59d006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 shutdown_connections 2024-03-29T04:10:09.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.656+0000 7f59d584a640 1 -- 172.21.15.181:0/172935786 wait complete. 2024-03-29T04:10:09.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T04:10:09.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 Processor -- start 2024-03-29T04:10:09.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 -- start start 2024-03-29T04:10:09.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 0x7f4ff0169510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff00d2ff0 con 0x7f4ff00ba090 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff00d32d0 con 0x7f4ff00b5dc0 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.770+0000 7f4ff6ed3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff00d2d90 con 0x7f4ff0167120 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4ff5470640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 0x7f4ff0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4fe7fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4ff00ba090 0x7f4ff00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48112/0 (socket says 172.21.15.181:48112) 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4fe7fff640 1 -- 172.21.15.181:0/2230323340 learned_addr learned my addr 172.21.15.181:0/2230323340 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440968672 0 0) 0x7f4ff00d32d0 con 0x7f4ff00b5dc0 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fd8003520 con 0x7f4ff00b5dc0 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1579200718 0 0) 0x7f4ff00d2ff0 con 0x7f4ff00ba090 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4ff5470640 1 -- 172.21.15.181:0/2230323340 >> v1:172.21.15.187:6789/0 conn(0x7f4ff00b5dc0 legacy=0x7f4ff00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4ff5470640 1 -- 172.21.15.181:0/2230323340 >> v1:172.21.15.181:6789/0 conn(0x7f4ff00ba090 legacy=0x7f4ff00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.771+0000 7f4ff5470640 1 -- 172.21.15.181:0/2230323340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff00d35b0 con 0x7f4ff0167120 2024-03-29T04:10:09.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff5470640 1 --2- 172.21.15.181:0/2230323340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 0x7f4ff0169510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009df0 tx=0x7f4fe802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d9a01acfd8bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8039070 con 0x7f4ff0167120 2024-03-29T04:10:09.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe802ccb0 con 0x7f4ff0167120 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4fe77fe640 1 -- 172.21.15.181:0/2230323340 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe802ce60 con 0x7f4ff0167120 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/2230323340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 msgr2=0x7f4ff0169510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/2230323340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 0x7f4ff0169510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009df0 tx=0x7f4fe802bf00 comp rx=0 tx=0).stop 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/2230323340 shutdown_connections 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/2230323340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff0167120 0x7f4ff0169510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/2230323340 >> 172.21.15.181:0/2230323340 conn(0x7f4ff00678f0 msgr2=0x7f4ff00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/2230323340 shutdown_connections 2024-03-29T04:10:09.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.772+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/2230323340 wait complete. 2024-03-29T04:10:09.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.773+0000 7f4ff6ed3640 1 Processor -- start 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 -- start start 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff00b5dc0 0x7f4ff0156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 0x7f4ff00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff00d4090 con 0x7f4ff0167120 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff00d3b70 con 0x7f4ff00ba090 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff6ed3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff00b83e0 con 0x7f4ff00b5dc0 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.774+0000 7f4ff5470640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33888/0 (socket says 172.21.15.181:33888) 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 -- 172.21.15.181:0/1402278210 learned_addr learned my addr 172.21.15.181:0/1402278210 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4fe7fff640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 0x7f4ff00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff4c6f640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff00b5dc0 0x7f4ff0156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 -- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff00b5dc0 msgr2=0x7f4ff0156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff00b5dc0 0x7f4ff0156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 -- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 msgr2=0x7f4ff00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 0x7f4ff00b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 -- 172.21.15.181:0/1402278210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff00b5060 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4fe7fff640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 0x7f4ff00b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff5470640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8037040 tx=0x7f4fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8039050 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe8045070 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe802ce60 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff0166c30 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.775+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff015ea60 con 0x7f4ff0167120 2024-03-29T04:10:09.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.776+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff0075890 con 0x7f4ff0167120 2024-03-29T04:10:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.777+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4fe803e850 con 0x7f4ff0167120 2024-03-29T04:10:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.777+0000 7f4fe57fa640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 0x7f4fdc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.777+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4fe80ae6b0 con 0x7f4ff0167120 2024-03-29T04:10:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.777+0000 7f4ff4c6f640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 0x7f4fdc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:09.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.778+0000 7f4ff4c6f640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 0x7f4fdc06e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0011e0 tx=0x7f4fec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:09.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.782+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4fe807c430 con 0x7f4ff0167120 2024-03-29T04:10:09.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.956+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4ff00d4bc0 con 0x7f4ff0167120 2024-03-29T04:10:09.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.957+0000 7f4fe57fa640 1 -- 172.21.15.181:0/1402278210 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f4fe8080730 con 0x7f4ff0167120 2024-03-29T04:10:09.936 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:10:09.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.958+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 msgr2=0x7f4fdc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.938 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.958+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 0x7f4fdc06e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0011e0 tx=0x7f4fec009040 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 msgr2=0x7f4ff00b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8037040 tx=0x7f4fe8004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 shutdown_connections 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ff0167120 0x7f4ff00b41e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ff00ba090 0x7f4ff00b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4fdc06c530 0x7f4fdc06e9f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 --2- 172.21.15.181:0/1402278210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ff00b5dc0 0x7f4ff0156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 >> 172.21.15.181:0/1402278210 conn(0x7f4ff00678f0 msgr2=0x7f4ff00d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 shutdown_connections 2024-03-29T04:10:09.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:09.959+0000 7f4ff6ed3640 1 -- 172.21.15.181:0/1402278210 wait complete. 2024-03-29T04:10:09.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:10.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.070+0000 7f7e8339d640 1 Processor -- start 2024-03-29T04:10:10.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.070+0000 7f7e8339d640 1 -- start start 2024-03-29T04:10:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8339d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8339d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e7c09b140 0x7f7e7c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8339d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e7c0c5c10 con 0x7f7e7c0cfbf0 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8339d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7c15c840 con 0x7f7e7c09b140 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8339d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7c0d0080 con 0x7f7e7c15c460 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8239b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e82b9c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e7c09b140 0x7f7e7c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e81b9a640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e7c0cfbf0 0x7f7e7c09aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48120/0 (socket says 172.21.15.181:48120) 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8239b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:46328/0 (socket says 172.21.15.181:46328) 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e81b9a640 1 -- 172.21.15.181:0/1768436234 learned_addr learned my addr 172.21.15.181:0/1768436234 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e82b9c640 1 -- 172.21.15.181:0/1768436234 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 msgr2=0x7f7e7c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e82b9c640 1 --2- 172.21.15.181:0/1768436234 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e82b9c640 1 -- 172.21.15.181:0/1768436234 >> v1:172.21.15.181:6789/0 conn(0x7f7e7c0cfbf0 legacy=0x7f7e7c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e82b9c640 1 -- 172.21.15.181:0/1768436234 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c0c5eb0 con 0x7f7e7c09b140 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e81399640 1 -- 172.21.15.181:0/1768436234 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681945597 0 0) 0x7f7e7c0c5c10 con 0x7f7e7c0cfbf0 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.071+0000 7f7e8239b640 1 --2- 172.21.15.181:0/1768436234 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:10.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.072+0000 7f7e82b9c640 1 --2- 172.21.15.181:0/1768436234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0c66b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0727f0 tx=0x7f7e7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa50ce5fb51cb5f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.072+0000 7f7e81399640 1 -- 172.21.15.181:0/1768436234 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74039070 con 0x7f7e7c09b140 2024-03-29T04:10:10.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.072+0000 7f7e81399640 1 -- 172.21.15.181:0/1768436234 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7402cc80 con 0x7f7e7c09b140 2024-03-29T04:10:10.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.072+0000 7f7e81399640 1 -- 172.21.15.181:0/1768436234 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74034660 con 0x7f7e7c09b140 2024-03-29T04:10:10.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 -- 172.21.15.181:0/1768436234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 msgr2=0x7f7e7c0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 --2- 172.21.15.181:0/1768436234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0c66b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0727f0 tx=0x7f7e7402bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 -- 172.21.15.181:0/1768436234 shutdown_connections 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 --2- 172.21.15.181:0/1768436234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0c66b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 --2- 172.21.15.181:0/1768436234 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c15c460 0x7f7e7c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 -- 172.21.15.181:0/1768436234 >> 172.21.15.181:0/1768436234 conn(0x7f7e7c0b3b10 msgr2=0x7f7e7c0b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 -- 172.21.15.181:0/1768436234 shutdown_connections 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.073+0000 7f7e8339d640 1 -- 172.21.15.181:0/1768436234 wait complete. 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.074+0000 7f7e8339d640 1 Processor -- start 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.074+0000 7f7e8339d640 1 -- start start 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15c460 0x7f7e7c1f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e7c09bfe0 0x7f7e7c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c09b720 con 0x7f7e7c09b140 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c0d0360 con 0x7f7e7c09bfe0 2024-03-29T04:10:10.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7c09b8a0 con 0x7f7e7c15c460 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33912/0 (socket says 172.21.15.181:33912) 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 -- 172.21.15.181:0/2375385801 learned_addr learned my addr 172.21.15.181:0/2375385801 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e82b9c640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e7c09bfe0 0x7f7e7c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e81b9a640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15c460 0x7f7e7c1f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 -- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15c460 msgr2=0x7f7e7c1f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15c460 0x7f7e7c1f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 -- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e7c09bfe0 msgr2=0x7f7e7c1f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e7c09bfe0 0x7f7e7c1f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 -- 172.21.15.181:0/2375385801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c0b61b0 con 0x7f7e7c09b140 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8239b640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e960 tx=0x7f7e7800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78017070 con 0x7f7e7c09b140 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7800ca00 con 0x7f7e7c09b140 2024-03-29T04:10:10.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c0b6490 con 0x7f7e7c09b140 2024-03-29T04:10:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cdd0 con 0x7f7e7c09b140 2024-03-29T04:10:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.075+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c0b6a50 con 0x7f7e7c09b140 2024-03-29T04:10:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.076+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7c075890 con 0x7f7e7c09b140 2024-03-29T04:10:10.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.077+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7e7801a070 con 0x7f7e7c09b140 2024-03-29T04:10:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.077+0000 7f7e72ffd640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 0x7f7e6006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.077+0000 7f7e81b9a640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 0x7f7e6006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.077+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7e7808bbf0 con 0x7f7e7c09b140 2024-03-29T04:10:10.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.078+0000 7f7e81b9a640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 0x7f7e6006ea20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c15ada0 tx=0x7f7e6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.082+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7e780597c0 con 0x7f7e7c09b140 2024-03-29T04:10:10.196 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:10.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:10.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.299+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e7c0b7ea0 con 0x7f7e7c09b140 2024-03-29T04:10:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.299+0000 7f7e72ffd640 1 -- 172.21.15.181:0/2375385801 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f7e7805dac0 con 0x7f7e7c09b140 2024-03-29T04:10:10.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:10:10.279 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.301+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 msgr2=0x7f7e6006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.301+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 0x7f7e6006ea20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c15ada0 tx=0x7f7e6c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:10.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.301+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 msgr2=0x7f7e7c0b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.301+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e960 tx=0x7f7e7800ee20 comp rx=0 tx=0).stop 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 shutdown_connections 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e7c09bfe0 0x7f7e7c1f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f7e6006c560 0x7f7e6006ea20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15c460 0x7f7e7c1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 --2- 172.21.15.181:0/2375385801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e7c09b140 0x7f7e7c0b5830 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 >> 172.21.15.181:0/2375385801 conn(0x7f7e7c0b3b10 msgr2=0x7f7e7c0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 shutdown_connections 2024-03-29T04:10:10.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.302+0000 7f7e8339d640 1 -- 172.21.15.181:0/2375385801 wait complete. 2024-03-29T04:10:10.292 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.455+0000 7f56329a8640 1 Processor -- start 2024-03-29T04:10:10.435 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.455+0000 7f56329a8640 1 -- start start 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f56329a8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f562c15e950 0x7f562c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f56329a8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 0x7f562c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f56329a8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f562c0d36f0 con 0x7f562c0d48c0 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f56329a8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f562c0d3cb0 con 0x7f562c15e950 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f56329a8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f562c0d4d00 con 0x7f562c0d30d0 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f562bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f562c15e950 0x7f562c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f562bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f562c15e950 0x7f562c0d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33920/0 (socket says 172.21.15.181:33920) 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f562bfff640 1 -- 172.21.15.181:0/1752253225 learned_addr learned my addr 172.21.15.181:0/1752253225 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.456+0000 7f5630f45640 1 --2- 172.21.15.181:0/1752253225 >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 0x7f562c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562bfff640 1 -- 172.21.15.181:0/1752253225 >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 msgr2=0x7f562c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562bfff640 1 --2- 172.21.15.181:0/1752253225 >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 0x7f562c0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.437 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562bfff640 1 -- 172.21.15.181:0/1752253225 >> v1:172.21.15.187:6789/0 conn(0x7f562c0d48c0 legacy=0x7f562c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562bfff640 1 -- 172.21.15.181:0/1752253225 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c0d39d0 con 0x7f562c15e950 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f5630f45640 1 --2- 172.21.15.181:0/1752253225 >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 0x7f562c0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562bfff640 1 --2- 172.21.15.181:0/1752253225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c15e950 0x7f562c0d4380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f562c0d3e30 tx=0x7f561c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761c50d8a18fe97a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562affd640 1 -- 172.21.15.181:0/1752253225 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c02f840 con 0x7f562c15e950 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562affd640 1 -- 172.21.15.181:0/1752253225 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c02f9e0 con 0x7f562c15e950 2024-03-29T04:10:10.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.457+0000 7f562affd640 1 -- 172.21.15.181:0/1752253225 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c02fdb0 con 0x7f562c15e950 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 -- 172.21.15.181:0/1752253225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c15e950 msgr2=0x7f562c0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 --2- 172.21.15.181:0/1752253225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c15e950 0x7f562c0d4380 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f562c0d3e30 tx=0x7f561c02ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 -- 172.21.15.181:0/1752253225 shutdown_connections 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 --2- 172.21.15.181:0/1752253225 >> v2:172.21.15.187:3300/0 conn(0x7f562c0d30d0 0x7f562c0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 --2- 172.21.15.181:0/1752253225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c15e950 0x7f562c0d4380 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 -- 172.21.15.181:0/1752253225 >> 172.21.15.181:0/1752253225 conn(0x7f562c0676e0 msgr2=0x7f562c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 -- 172.21.15.181:0/1752253225 shutdown_connections 2024-03-29T04:10:10.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.458+0000 7f56329a8640 1 -- 172.21.15.181:0/1752253225 wait complete. 2024-03-29T04:10:10.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.459+0000 7f56329a8640 1 Processor -- start 2024-03-29T04:10:10.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.459+0000 7f56329a8640 1 -- start start 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f562c0d30d0 0x7f562c1f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f562c15e950 0x7f562c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f562c15ed30 con 0x7f562c0d48c0 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0b7da0 con 0x7f562c15e950 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f56329a8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f562c15eeb0 con 0x7f562c0d30d0 2024-03-29T04:10:10.442 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f562b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f562b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33928/0 (socket says 172.21.15.181:33928) 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f562b7fe640 1 -- 172.21.15.181:0/148980745 learned_addr learned my addr 172.21.15.181:0/148980745 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f562bfff640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f562c0d30d0 0x7f562c1f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.460+0000 7f5630f45640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f562c15e950 0x7f562c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 -- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f562c0d30d0 msgr2=0x7f562c1f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f562c0d30d0 0x7f562c1f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 -- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f562c15e950 msgr2=0x7f562c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f562c15e950 0x7f562c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 -- 172.21.15.181:0/148980745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1f4600 con 0x7f562c0d48c0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f562b7fe640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f561400c400 tx=0x7f561400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140052d0 con 0x7f562c0d48c0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5614005490 con 0x7f562c0d48c0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1f4870 con 0x7f562c0d48c0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614007260 con 0x7f562c0d48c0 2024-03-29T04:10:10.443 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.461+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1f4e30 con 0x7f562c0d48c0 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.462+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c075a10 con 0x7f562c0d48c0 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.463+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5614007400 con 0x7f562c0d48c0 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.463+0000 7f5628ff9640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 0x7f560c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.463+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f561408afb0 con 0x7f562c0d48c0 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.463+0000 7f562bfff640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 0x7f560c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.463+0000 7f562bfff640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 0x7f560c06e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f561c036000 tx=0x7f561c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.450 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.471+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5614058b80 con 0x7f562c0d48c0 2024-03-29T04:10:10.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.656+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f562c0d1280 con 0x7f562c0d48c0 2024-03-29T04:10:10.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.657+0000 7f5628ff9640 1 -- 172.21.15.181:0/148980745 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7f561405ce80 con 0x7f562c0d48c0 2024-03-29T04:10:10.637 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:10.637 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:10.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.659+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 msgr2=0x7f560c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.659+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 0x7f560c06e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f561c036000 tx=0x7f561c008030 comp rx=0 tx=0).stop 2024-03-29T04:10:10.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.659+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 msgr2=0x7f562c158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.659+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f561400c400 tx=0x7f561400c8c0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 shutdown_connections 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f562c15e950 0x7f562c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f562c0d48c0 0x7f562c158320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f560c06c530 0x7f560c06e9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 --2- 172.21.15.181:0/148980745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f562c0d30d0 0x7f562c1f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 >> 172.21.15.181:0/148980745 conn(0x7f562c0676e0 msgr2=0x7f562c0b8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 shutdown_connections 2024-03-29T04:10:10.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.660+0000 7f56329a8640 1 -- 172.21.15.181:0/148980745 wait complete. 2024-03-29T04:10:10.651 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:10:10.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:10.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.775+0000 7f06bc675640 1 Processor -- start 2024-03-29T04:10:10.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.775+0000 7f06bc675640 1 -- start start 2024-03-29T04:10:10.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bc675640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06b40cd3b0 0x7f06b40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bc675640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b40cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bc675640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b40724e0 con 0x7f06b415a960 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bc675640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f06b40730f0 con 0x7f06b40cd3b0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bc675640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06b40727c0 con 0x7f06b40cb3b0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06b40cd3b0 0x7f06b40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06b40cd3b0 0x7f06b40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33934/0 (socket says 172.21.15.181:33934) 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 -- 172.21.15.181:0/3426874051 learned_addr learned my addr 172.21.15.181:0/3426874051 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06bac12640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b40cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 -- 172.21.15.181:0/3426874051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 msgr2=0x7f06b40cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b40cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 -- 172.21.15.181:0/3426874051 >> v1:172.21.15.187:6789/0 conn(0x7f06b415a960 legacy=0x7f06b4071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.776+0000 7f06b9c10640 1 -- 172.21.15.181:0/3426874051 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b4072aa0 con 0x7f06b40cd3b0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.777+0000 7f06b9c10640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b40cae70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009870 tx=0x7f06a402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e9932ac5d708c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.777+0000 7f06b940f640 1 -- 172.21.15.181:0/3426874051 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4034070 con 0x7f06b40cd3b0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.777+0000 7f06b940f640 1 -- 172.21.15.181:0/3426874051 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4030840 con 0x7f06b40cd3b0 2024-03-29T04:10:10.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.777+0000 7f06b940f640 1 -- 172.21.15.181:0/3426874051 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4030c10 con 0x7f06b40cd3b0 2024-03-29T04:10:10.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 -- 172.21.15.181:0/3426874051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 msgr2=0x7f06b40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b40cae70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009870 tx=0x7f06a402b980 comp rx=0 tx=0).stop 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 -- 172.21.15.181:0/3426874051 shutdown_connections 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b40cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 --2- 172.21.15.181:0/3426874051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b40cae70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 -- 172.21.15.181:0/3426874051 >> 172.21.15.181:0/3426874051 conn(0x7f06b40b1c90 msgr2=0x7f06b40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 -- 172.21.15.181:0/3426874051 shutdown_connections 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.778+0000 7f06bc675640 1 -- 172.21.15.181:0/3426874051 wait complete. 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.779+0000 7f06bc675640 1 Processor -- start 2024-03-29T04:10:10.759 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.779+0000 7f06bc675640 1 -- start start 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b4159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06b415a960 0x7f06b40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4072180 con 0x7f06b40cd3b0 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06b415ad40 con 0x7f06b415a960 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bc675640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06b40cca40 con 0x7f06b40cb3b0 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33940/0 (socket says 172.21.15.181:33940) 2024-03-29T04:10:10.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 -- 172.21.15.181:0/870230045 learned_addr learned my addr 172.21.15.181:0/870230045 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06ba411640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b4159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06bac12640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06b415a960 0x7f06b40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 -- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 msgr2=0x7f06b4159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b4159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 -- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06b415a960 msgr2=0x7f06b40c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06b415a960 0x7f06b40c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 -- 172.21.15.181:0/870230045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b4152d60 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06b9c10640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000c00 tx=0x7f06a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.780+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4048070 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.781+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4030640 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.781+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4034040 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.781+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b41530a0 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.781+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b41538e0 con 0x7f06b40cd3b0 2024-03-29T04:10:10.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.781+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b4075a10 con 0x7f06b40cd3b0 2024-03-29T04:10:10.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.782+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f06a404b070 con 0x7f06b40cd3b0 2024-03-29T04:10:10.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.783+0000 7f06aaffd640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 0x7f069806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:10.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.783+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f06a40ad8a0 con 0x7f06b40cd3b0 2024-03-29T04:10:10.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.783+0000 7f06ba411640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 0x7f069806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:10.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.783+0000 7f06ba411640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 0x7f069806e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f06b00016e0 tx=0x7f06b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:10.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:10.787+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f06b41530a0 con 0x7f06b40cd3b0 2024-03-29T04:10:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.021+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b4167110 con 0x7f06b40cd3b0 2024-03-29T04:10:11.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.022+0000 7f06aaffd640 1 -- 172.21.15.181:0/870230045 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f06a4030810 con 0x7f06b40cd3b0 2024-03-29T04:10:11.002 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:11.002 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:10:11.002 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 msgr2=0x7f069806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 0x7f069806e9f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f06b00016e0 tx=0x7f06b0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 msgr2=0x7f06b4153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000c00 tx=0x7f06a4004730 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 shutdown_connections 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06b415a960 0x7f06b40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b40cd3b0 0x7f06b4153e00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f069806c530 0x7f069806e9f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 --2- 172.21.15.181:0/870230045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06b40cb3b0 0x7f06b4159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 >> 172.21.15.181:0/870230045 conn(0x7f06b40b1c90 msgr2=0x7f06b4071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 shutdown_connections 2024-03-29T04:10:11.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.024+0000 7f06bc675640 1 -- 172.21.15.181:0/870230045 wait complete. 2024-03-29T04:10:11.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:11.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.136+0000 7f7a0b349640 1 Processor -- start 2024-03-29T04:10:11.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.136+0000 7f7a0b349640 1 -- start start 2024-03-29T04:10:11.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a0b349640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a0415e950 0x7f7a040d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a0b349640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a040d48c0 0x7f7a040d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a0b349640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a040d36f0 con 0x7f7a040d2f00 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a0b349640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f7a040d3cb0 con 0x7f7a0415e950 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a0b349640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7a040d4d00 con 0x7f7a040d48c0 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a0415e950 0x7f7a040d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a0415e950 0x7f7a040d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33950/0 (socket says 172.21.15.181:33950) 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 -- 172.21.15.181:0/559914933 learned_addr learned my addr 172.21.15.181:0/559914933 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a088e4640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a040d48c0 0x7f7a040d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 -- 172.21.15.181:0/559914933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a040d48c0 msgr2=0x7f7a040d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a040d48c0 0x7f7a040d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 -- 172.21.15.181:0/559914933 >> v1:172.21.15.187:6789/0 conn(0x7f7a040d2f00 legacy=0x7f7a040cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.137+0000 7f7a090e5640 1 -- 172.21.15.181:0/559914933 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a040d39d0 con 0x7f7a0415e950 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.138+0000 7f7a090e5640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0415e950 0x7f7a040d4380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009870 tx=0x7f79f402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec77e365b40b011 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.138+0000 7f79fbfff640 1 -- 172.21.15.181:0/559914933 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f402f840 con 0x7f7a0415e950 2024-03-29T04:10:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.138+0000 7f79fbfff640 1 -- 172.21.15.181:0/559914933 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f402f9e0 con 0x7f7a0415e950 2024-03-29T04:10:11.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.138+0000 7f79fbfff640 1 -- 172.21.15.181:0/559914933 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f402fdb0 con 0x7f7a0415e950 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 -- 172.21.15.181:0/559914933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0415e950 msgr2=0x7f7a040d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0415e950 0x7f7a040d4380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009870 tx=0x7f79f402ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 -- 172.21.15.181:0/559914933 shutdown_connections 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a040d48c0 0x7f7a040d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 --2- 172.21.15.181:0/559914933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a0415e950 0x7f7a040d4380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 -- 172.21.15.181:0/559914933 >> 172.21.15.181:0/559914933 conn(0x7f7a040676e0 msgr2=0x7f7a04067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 -- 172.21.15.181:0/559914933 shutdown_connections 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.139+0000 7f7a0b349640 1 -- 172.21.15.181:0/559914933 wait complete. 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.140+0000 7f7a0b349640 1 Processor -- start 2024-03-29T04:10:11.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.140+0000 7f7a0b349640 1 -- start start 2024-03-29T04:10:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a040d48c0 0x7f7a040ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a0415e950 0x7f7a040ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7a040d3390 con 0x7f7a040d2f00 2024-03-29T04:10:11.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7a040b7da0 con 0x7f7a040d48c0 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a0b349640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0415ed30 con 0x7f7a0415e950 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33954/0 (socket says 172.21.15.181:33954) 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 -- 172.21.15.181:0/3903359720 learned_addr learned my addr 172.21.15.181:0/3903359720 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a088e4640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a040d48c0 0x7f7a040ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a098e6640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a0415e950 0x7f7a040ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 -- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a0415e950 msgr2=0x7f7a040ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a0415e950 0x7f7a040ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 -- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a040d48c0 msgr2=0x7f7a040ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a040d48c0 0x7f7a040ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 -- 172.21.15.181:0/3903359720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0414ce70 con 0x7f7a040d2f00 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f7a090e5640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f79f4000c00 tx=0x7f79f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.141+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4037bc0 con 0x7f7a040d2f00 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.142+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f4046070 con 0x7f7a040d2f00 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.142+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4041400 con 0x7f7a040d2f00 2024-03-29T04:10:11.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.142+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a040c8bc0 con 0x7f7a040d2f00 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.142+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a040c9180 con 0x7f7a040d2f00 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.143+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a04075a10 con 0x7f7a040d2f00 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.143+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f79f4040070 con 0x7f7a040d2f00 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.144+0000 7f79f9ffb640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 0x7f79e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.144+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f79f40adff0 con 0x7f7a040d2f00 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.144+0000 7f7a088e4640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 0x7f79e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.144+0000 7f7a088e4640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 0x7f79e806e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cdc10 tx=0x7f7a00009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.152+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f79f4033030 con 0x7f7a040d2f00 2024-03-29T04:10:11.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.368+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a040b69a0 con 0x7f7a040d2f00 2024-03-29T04:10:11.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.368+0000 7f79f9ffb640 1 -- 172.21.15.181:0/3903359720 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f79f407bd70 con 0x7f7a040d2f00 2024-03-29T04:10:11.348 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:11.348 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2033846831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2033846831},{"type":"v1","addr":"172.21.15.181:6837","nonce":2033846831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/285659982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":285659982},{"type":"v1","addr":"172.21.15.187:6834","nonce":285659982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3206529302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3206529302},{"type":"v1","addr":"172.21.15.181:6835","nonce":3206529302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:09:15.639711+0000","modified":"2024-03-29T04:09:31.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6835/3068825067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3068825067},{"type":"v1","addr":"172.21.15.187:6835","nonce":3068825067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-03-29T04:10:11.348 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 9 2024-03-29T04:10:11.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 msgr2=0x7f79e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 0x7f79e806e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cdc10 tx=0x7f7a00009190 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 msgr2=0x7f7a0414c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f79f4000c00 tx=0x7f79f4004730 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 shutdown_connections 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79e806c530 0x7f79e806e9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a0415e950 0x7f7a040ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a040d48c0 0x7f7a040ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 --2- 172.21.15.181:0/3903359720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a040d2f00 0x7f7a0414c930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 >> 172.21.15.181:0/3903359720 conn(0x7f7a040676e0 msgr2=0x7f7a0415dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 shutdown_connections 2024-03-29T04:10:11.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.371+0000 7f7a0b349640 1 -- 172.21.15.181:0/3903359720 wait complete. 2024-03-29T04:10:11.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.485+0000 7fd7dccb2640 1 Processor -- start 2024-03-29T04:10:11.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.485+0000 7fd7dccb2640 1 -- start start 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7dccb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7dccb2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d80b5000 con 0x7fd7d815e450 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7dccb2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d80b4d20 con 0x7fd7d8162ce0 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7dccb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d8162850 con 0x7fd7d80b0780 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d659c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd7d815e450 0x7fd7d80b41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48136/0 (socket says 172.21.15.181:48136) 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d659c640 1 -- 172.21.15.181:0/3725434359 learned_addr learned my addr 172.21.15.181:0/3725434359 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d6d9d640 1 --2- 172.21.15.181:0/3725434359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063499741 0 0) 0x7fd7d80b5000 con 0x7fd7d815e450 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7c4003520 con 0x7fd7d815e450 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973662936 0 0) 0x7fd7d80b4d20 con 0x7fd7d8162ce0 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7d80b5000 con 0x7fd7d8162ce0 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.486+0000 7fd7d6d9d640 1 -- 172.21.15.181:0/3725434359 >> v1:172.21.15.187:6789/0 conn(0x7fd7d8162ce0 legacy=0x7fd7d80b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d6d9d640 1 -- 172.21.15.181:0/3725434359 >> v1:172.21.15.181:6789/0 conn(0x7fd7d815e450 legacy=0x7fd7d80b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d6d9d640 1 -- 172.21.15.181:0/3725434359 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d8162200 con 0x7fd7d80b0780 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d6d9d640 1 --2- 172.21.15.181:0/3725434359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d80d0e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7d81629d0 tx=0x7fd7d002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ad71dc0369798 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0039070 con 0x7fd7d80b0780 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d002ccd0 con 0x7fd7d80b0780 2024-03-29T04:10:11.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.487+0000 7fd7d5d9b640 1 -- 172.21.15.181:0/3725434359 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0034640 con 0x7fd7d80b0780 2024-03-29T04:10:11.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 -- 172.21.15.181:0/3725434359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 msgr2=0x7fd7d80d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/3725434359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d80d0e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7d81629d0 tx=0x7fd7d002bf20 comp rx=0 tx=0).stop 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 -- 172.21.15.181:0/3725434359 shutdown_connections 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/3725434359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d80d0e40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 -- 172.21.15.181:0/3725434359 >> 172.21.15.181:0/3725434359 conn(0x7fd7d806c690 msgr2=0x7fd7d806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 -- 172.21.15.181:0/3725434359 shutdown_connections 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.488+0000 7fd7dccb2640 1 -- 172.21.15.181:0/3725434359 wait complete. 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.490+0000 7fd7dccb2640 1 Processor -- start 2024-03-29T04:10:11.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.490+0000 7fd7dccb2640 1 -- start start 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 0x7fd7d814dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d815e830 con 0x7fd7d8162ce0 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d806ed20 con 0x7fd7d815e450 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7dccb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d815e9b0 con 0x7fd7d80b0780 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33960/0 (socket says 172.21.15.181:33960) 2024-03-29T04:10:11.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d6d9d640 1 -- 172.21.15.181:0/1845557830 learned_addr learned my addr 172.21.15.181:0/1845557830 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7cffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 0x7fd7d814dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 -- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 msgr2=0x7fd7d814dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 0x7fd7d814dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 -- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 msgr2=0x7fd7d81f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 -- 172.21.15.181:0/1845557830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d815ee30 con 0x7fd7d80b0780 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d6d9d640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7cffff640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 0x7fd7d814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7d659c640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d815a280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c410 tx=0x7fd7c800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.491+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8017070 con 0x7fd7d80b0780 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.492+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c8005ce0 con 0x7fd7d80b0780 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.492+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d815f110 con 0x7fd7d80b0780 2024-03-29T04:10:11.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.492+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80072d0 con 0x7fd7d80b0780 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.492+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d815f6d0 con 0x7fd7d80b0780 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.493+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d8075a10 con 0x7fd7d80b0780 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.493+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd7c801a070 con 0x7fd7d80b0780 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.493+0000 7fd7ceffd640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 0x7fd7b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.493+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd7c808ad00 con 0x7fd7d80b0780 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.493+0000 7fd7cffff640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 0x7fd7b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.494+0000 7fd7cffff640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 0x7fd7b806e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0007fd0 tx=0x7fd7c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.498+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd7c8058960 con 0x7fd7d80b0780 2024-03-29T04:10:11.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.676+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7d806e860 con 0x7fd7d80b0780 2024-03-29T04:10:11.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.677+0000 7fd7ceffd640 1 -- 172.21.15.181:0/1845557830 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18913 (secure 0 0 0) 0x7fd7c805cc60 con 0x7fd7d80b0780 2024-03-29T04:10:11.657 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:11.657 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":27,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:09:57.426071+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:09:13.613655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T04:09:14.603835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T04:09:18.323786+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 msgr2=0x7fd7b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 0x7fd7b806e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0007fd0 tx=0x7fd7c0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 msgr2=0x7fd7d815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d815a280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c410 tx=0x7fd7c800c8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 shutdown_connections 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7d8162ce0 0x7fd7d81f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d815e450 0x7fd7d814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd7b806c530 0x7fd7b806e9f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 --2- 172.21.15.181:0/1845557830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7d80b0780 0x7fd7d815a280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 >> 172.21.15.181:0/1845557830 conn(0x7fd7d806c690 msgr2=0x7fd7d8158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 shutdown_connections 2024-03-29T04:10:11.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.679+0000 7fd7dccb2640 1 -- 172.21.15.181:0/1845557830 wait complete. 2024-03-29T04:10:11.670 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:10:11.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.791+0000 7f37a46b7640 1 Processor -- start 2024-03-29T04:10:11.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.791+0000 7f37a46b7640 1 -- start start 2024-03-29T04:10:11.772 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.792+0000 7f37a46b7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f379c15e7a0 0x7f379c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.792+0000 7f37a46b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f379c0d19c0 0x7f379c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.792+0000 7f37a46b7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379c0d3120 con 0x7f379c0cec60 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.792+0000 7f37a46b7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f379c0cfc40 con 0x7f379c15e7a0 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.792+0000 7f37a46b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f379c0d3400 con 0x7f379c0d19c0 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f379c15e7a0 0x7f379c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f379c15e7a0 0x7f379c0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33970/0 (socket says 172.21.15.181:33970) 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 -- 172.21.15.181:0/166562198 learned_addr learned my addr 172.21.15.181:0/166562198 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2c54640 1 --2- 172.21.15.181:0/166562198 >> v2:172.21.15.187:3300/0 conn(0x7f379c0d19c0 0x7f379c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 -- 172.21.15.181:0/166562198 >> v2:172.21.15.187:3300/0 conn(0x7f379c0d19c0 msgr2=0x7f379c0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 --2- 172.21.15.181:0/166562198 >> v2:172.21.15.187:3300/0 conn(0x7f379c0d19c0 0x7f379c0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 -- 172.21.15.181:0/166562198 >> v1:172.21.15.187:6789/0 conn(0x7f379c0cec60 legacy=0x7f379c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.793+0000 7f37a2453640 1 -- 172.21.15.181:0/166562198 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c0d36e0 con 0x7f379c15e7a0 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a2453640 1 --2- 172.21.15.181:0/166562198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c15e7a0 0x7f379c0cf3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f379c0cf320 tx=0x7f378c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f64989f8d39670 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a1451640 1 -- 172.21.15.181:0/166562198 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c035020 con 0x7f379c15e7a0 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a1451640 1 -- 172.21.15.181:0/166562198 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c031870 con 0x7f379c15e7a0 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a1451640 1 -- 172.21.15.181:0/166562198 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c031c40 con 0x7f379c15e7a0 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 -- 172.21.15.181:0/166562198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c15e7a0 msgr2=0x7f379c0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 --2- 172.21.15.181:0/166562198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c15e7a0 0x7f379c0cf3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f379c0cf320 tx=0x7f378c02b910 comp rx=0 tx=0).stop 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 -- 172.21.15.181:0/166562198 shutdown_connections 2024-03-29T04:10:11.777 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 --2- 172.21.15.181:0/166562198 >> v2:172.21.15.187:3300/0 conn(0x7f379c0d19c0 0x7f379c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 --2- 172.21.15.181:0/166562198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c15e7a0 0x7f379c0cf3f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 -- 172.21.15.181:0/166562198 >> 172.21.15.181:0/166562198 conn(0x7f379c0676b0 msgr2=0x7f379c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 -- 172.21.15.181:0/166562198 shutdown_connections 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.794+0000 7f37a46b7640 1 -- 172.21.15.181:0/166562198 wait complete. 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 Processor -- start 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 -- start start 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0cec60 0x7f379c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c149c50 0x7f379c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0d3ee0 con 0x7f379c0d19c0 2024-03-29T04:10:11.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0cf930 con 0x7f379c149c50 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a46b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f379c0d4060 con 0x7f379c0cec60 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a1c52640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a1c52640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33976/0 (socket says 172.21.15.181:33976) 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a1c52640 1 -- 172.21.15.181:0/3493993304 learned_addr learned my addr 172.21.15.181:0/3493993304 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a2c54640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c149c50 0x7f379c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.796+0000 7f37a2453640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0cec60 0x7f379c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 -- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0cec60 msgr2=0x7f379c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0cec60 0x7f379c0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 -- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c149c50 msgr2=0x7f379c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c149c50 0x7f379c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 -- 172.21.15.181:0/3493993304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c152fc0 con 0x7f379c0d19c0 2024-03-29T04:10:11.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a1c52640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f379800c440 tx=0x7f379800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980052d0 con 0x7f379c0d19c0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3798005470 con 0x7f379c0d19c0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c0ce6f0 con 0x7f379c0d19c0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798007260 con 0x7f379c0d19c0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.797+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c156e50 con 0x7f379c0d19c0 2024-03-29T04:10:11.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.798+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c075900 con 0x7f379c0d19c0 2024-03-29T04:10:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.798+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3798007400 con 0x7f379c0d19c0 2024-03-29T04:10:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.799+0000 7f3792ffd640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 0x7f378006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.799+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f379808ada0 con 0x7f379c0d19c0 2024-03-29T04:10:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.799+0000 7f37a2453640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 0x7f378006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:11.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.799+0000 7f37a2453640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 0x7f378006ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f378c036000 tx=0x7f378c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:11.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:11.803+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f379c152fc0 con 0x7f379c0d19c0 2024-03-29T04:10:11.996 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.017+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f379c06d340 con 0x7f379c0d19c0 2024-03-29T04:10:11.996 INFO:tasks.ceph.mon.a.smithi181.stderr:2024-03-29T04:10:12.017+0000 7f8224f14640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:10:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.018+0000 7f3792ffd640 1 -- 172.21.15.181:0/3493993304 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f3798058970 con 0x7f379c0d19c0 2024-03-29T04:10:11.998 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:10:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 msgr2=0x7f378006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 0x7f378006ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f378c036000 tx=0x7f378c03c040 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 msgr2=0x7f379c1f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f379800c440 tx=0x7f379800c900 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 shutdown_connections 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f379c149c50 0x7f379c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f379c0d19c0 0x7f379c1f3e70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f378006c560 0x7f378006ea20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 --2- 172.21.15.181:0/3493993304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f379c0cec60 0x7f379c0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 >> 172.21.15.181:0/3493993304 conn(0x7f379c0676b0 msgr2=0x7f379c0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 shutdown_connections 2024-03-29T04:10:12.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.022+0000 7f37a46b7640 1 -- 172.21.15.181:0/3493993304 wait complete. 2024-03-29T04:10:12.004 INFO:tasks.ceph.mds.b.smithi187.stderr: -5273> 2024-03-29T04:09:20.179+0000 7f8772d76640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:10:12.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:10:12.034 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-03-29T04:10:12.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 Processor -- start 2024-03-29T04:10:12.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 -- start start 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e70073310 0x7f0e7006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e700d1be0 0x7f0e7006bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e70073880 con 0x7f0e7015e5b0 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f0e700d0e20 con 0x7f0e700d1be0 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.135+0000 7f0e74a4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700d1100 con 0x7f0e70073310 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e700d1be0 0x7f0e7006bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e700d1be0 0x7f0e7006bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33984/0 (socket says 172.21.15.181:33984) 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 -- 172.21.15.181:0/1953531978 learned_addr learned my addr 172.21.15.181:0/1953531978 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e67fff640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e70073310 0x7f0e7006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 -- 172.21.15.181:0/1953531978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e70073310 msgr2=0x7f0e7006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e70073310 0x7f0e7006b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 -- 172.21.15.181:0/1953531978 >> v1:172.21.15.181:6789/0 conn(0x7f0e7015e5b0 legacy=0x7f0e70162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.136+0000 7f0e6ed9d640 1 -- 172.21.15.181:0/1953531978 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e700d13e0 con 0x7f0e700d1be0 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e6ed9d640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e700d1be0 0x7f0e7006bac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009e50 tx=0x7f0e6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ba9227beba2996 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e6dd9b640 1 -- 172.21.15.181:0/1953531978 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6802ca70 con 0x7f0e700d1be0 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e6dd9b640 1 -- 172.21.15.181:0/1953531978 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6802cc10 con 0x7f0e700d1be0 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e6dd9b640 1 -- 172.21.15.181:0/1953531978 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680343f0 con 0x7f0e700d1be0 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 -- 172.21.15.181:0/1953531978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e700d1be0 msgr2=0x7f0e7006bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e700d1be0 0x7f0e7006bac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009e50 tx=0x7f0e6802bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 -- 172.21.15.181:0/1953531978 shutdown_connections 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e700d1be0 0x7f0e7006bac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/1953531978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e70073310 0x7f0e7006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 -- 172.21.15.181:0/1953531978 >> 172.21.15.181:0/1953531978 conn(0x7f0e7006e720 msgr2=0x7f0e7006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:12.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 -- 172.21.15.181:0/1953531978 shutdown_connections 2024-03-29T04:10:12.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.137+0000 7f0e74a4a640 1 -- 172.21.15.181:0/1953531978 wait complete. 2024-03-29T04:10:12.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 Processor -- start 2024-03-29T04:10:12.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 -- start start 2024-03-29T04:10:12.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e700d1be0 0x7f0e7014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 0x7f0e700b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7006f9a0 con 0x7f0e70073310 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7015e990 con 0x7f0e7015e5b0 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e74a4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7006fb20 con 0x7f0e700d1be0 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e6e59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e6e59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33994/0 (socket says 172.21.15.181:33994) 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.139+0000 7f0e6e59c640 1 -- 172.21.15.181:0/2311800618 learned_addr learned my addr 172.21.15.181:0/2311800618 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e67fff640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e700d1be0 0x7f0e7014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6ed9d640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 0x7f0e700b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 -- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e700d1be0 msgr2=0x7f0e7014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e700d1be0 0x7f0e7014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 -- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 msgr2=0x7f0e700b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 0x7f0e700b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 -- 172.21.15.181:0/2311800618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7015b560 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6ed9d640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 0x7f0e700b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e6e59c640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c3c0 tx=0x7f0e6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600052d0 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e60005490 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e70160f60 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60007260 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.140+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e701614e0 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.141+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e700b03c0 con 0x7f0e70073310 2024-03-29T04:10:12.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.142+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0e60007400 con 0x7f0e70073310 2024-03-29T04:10:12.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.142+0000 7f0e66ffd640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 0x7f0e5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:12.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.142+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f0e6008afe0 con 0x7f0e70073310 2024-03-29T04:10:12.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.142+0000 7f0e67fff640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 0x7f0e5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:12.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.142+0000 7f0e67fff640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 0x7f0e5006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e70161d00 tx=0x7f0e58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:12.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.146+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0e700b03c0 con 0x7f0e70073310 2024-03-29T04:10:12.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:12.361+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0e70075890 con 0x7f0e70073310 2024-03-29T04:10:12.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.020+0000 7f0e66ffd640 1 -- 172.21.15.181:0/2311800618 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f0e600591a0 con 0x7f0e70073310 2024-03-29T04:10:13.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 msgr2=0x7f0e5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 0x7f0e5006e9f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e70161d00 tx=0x7f0e58008040 comp rx=0 tx=0).stop 2024-03-29T04:10:13.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 msgr2=0x7f0e7015ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c3c0 tx=0x7f0e6000c880 comp rx=0 tx=0).stop 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 shutdown_connections 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e7015e5b0 0x7f0e700b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0e5006c530 0x7f0e5006e9f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e700d1be0 0x7f0e7014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 --2- 172.21.15.181:0/2311800618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e70073310 0x7f0e7015ae60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.022+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 >> 172.21.15.181:0/2311800618 conn(0x7f0e7006e720 msgr2=0x7f0e700b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.023+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 shutdown_connections 2024-03-29T04:10:13.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.023+0000 7f0e74a4a640 1 -- 172.21.15.181:0/2311800618 wait complete. 2024-03-29T04:10:13.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.144+0000 7f3aaf990640 1 Processor -- start 2024-03-29T04:10:13.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.144+0000 7f3aaf990640 1 -- start start 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aaf990640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa813f2d0 0x7f3aa806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aaf990640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa8160120 con 0x7f3aa806d760 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aaf990640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa815d470 con 0x7f3aa815cfa0 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aaf990640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa806dbc0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa813f2d0 0x7f3aa806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3aa813f2d0 0x7f3aa806d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34006/0 (socket says 172.21.15.181:34006) 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 -- 172.21.15.181:0/668393878 learned_addr learned my addr 172.21.15.181:0/668393878 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aacf2b640 1 --1- 172.21.15.181:0/668393878 >> v1:172.21.15.181:6789/0 conn(0x7f3aa806d760 0x7f3aa815c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48148/0 (socket says 172.21.15.181:48148) 2024-03-29T04:10:13.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 -- 172.21.15.181:0/668393878 >> v1:172.21.15.187:6789/0 conn(0x7f3aa815cfa0 legacy=0x7f3aa80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 -- 172.21.15.181:0/668393878 >> v1:172.21.15.181:6789/0 conn(0x7f3aa806d760 legacy=0x7f3aa815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.145+0000 7f3aad72c640 1 -- 172.21.15.181:0/668393878 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa815d750 con 0x7f3aa813f2d0 2024-03-29T04:10:13.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3aad72c640 1 --2- 172.21.15.181:0/668393878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa806d220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009870 tx=0x7f3aa402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4076297b1a27a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3a9ffff640 1 -- 172.21.15.181:0/668393878 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa402f840 con 0x7f3aa813f2d0 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3a9ffff640 1 -- 172.21.15.181:0/668393878 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa402f9e0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3a9ffff640 1 -- 172.21.15.181:0/668393878 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa402fdb0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3aaf990640 1 -- 172.21.15.181:0/668393878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 msgr2=0x7f3aa806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.146+0000 7f3aaf990640 1 --2- 172.21.15.181:0/668393878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa806d220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009870 tx=0x7f3aa402ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.147+0000 7f3aaf990640 1 -- 172.21.15.181:0/668393878 shutdown_connections 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.147+0000 7f3aaf990640 1 --2- 172.21.15.181:0/668393878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa806d220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.147+0000 7f3aaf990640 1 -- 172.21.15.181:0/668393878 >> 172.21.15.181:0/668393878 conn(0x7f3aa806f060 msgr2=0x7f3aa806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.147+0000 7f3aaf990640 1 -- 172.21.15.181:0/668393878 shutdown_connections 2024-03-29T04:10:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.147+0000 7f3aaf990640 1 -- 172.21.15.181:0/668393878 wait complete. 2024-03-29T04:10:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 Processor -- start 2024-03-29T04:10:13.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 -- start start 2024-03-29T04:10:13.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa806d760 0x7f3aa81e7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa815cfa0 0x7f3aa81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa813f6b0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80cf140 con 0x7f3aa806d760 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.148+0000 7f3aaf990640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa80cea80 con 0x7f3aa815cfa0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34016/0 (socket says 172.21.15.181:34016) 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 -- 172.21.15.181:0/4207548364 learned_addr learned my addr 172.21.15.181:0/4207548364 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 -- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa815cfa0 msgr2=0x7f3aa81542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa815cfa0 0x7f3aa81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 -- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa806d760 msgr2=0x7f3aa81e7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa806d760 0x7f3aa81e7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 -- 172.21.15.181:0/4207548364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa81f4440 con 0x7f3aa813f2d0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aacf2b640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c4d0 tx=0x7f3a9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98018070 con 0x7f3aa813f2d0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a98007480 con 0x7f3aa813f2d0 2024-03-29T04:10:13.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980047e0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa8072a20 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.149+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa8072fe0 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.150+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa8075900 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.151+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a9801b070 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.151+0000 7f3a9dffb640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 0x7f3a8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.151+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f3a9808af10 con 0x7f3aa813f2d0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.151+0000 7f3aad72c640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 0x7f3a8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:13.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.151+0000 7f3aad72c640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 0x7f3a8c06e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4036000 tx=0x7f3aa4039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:13.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.155+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3aa81f4440 con 0x7f3aa813f2d0 2024-03-29T04:10:13.308 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:13.329+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3aa80d0de0 con 0x7f3aa813f2d0 2024-03-29T04:10:14.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.021+0000 7f3a9dffb640 1 -- 172.21.15.181:0/4207548364 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f3a98059040 con 0x7f3aa813f2d0 2024-03-29T04:10:14.001 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 msgr2=0x7f3a8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 0x7f3a8c06e9f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4036000 tx=0x7f3aa4039040 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 msgr2=0x7f3aa8153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c4d0 tx=0x7f3a9800c990 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 shutdown_connections 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3a8c06c530 0x7f3a8c06e9f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3aa815cfa0 0x7f3aa81542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3aa813f2d0 0x7f3aa8153d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 --2- 172.21.15.181:0/4207548364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3aa806d760 0x7f3aa81e7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 >> 172.21.15.181:0/4207548364 conn(0x7f3aa806f060 msgr2=0x7f3aa80b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 shutdown_connections 2024-03-29T04:10:14.005 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.025+0000 7f3aaf990640 1 -- 172.21.15.181:0/4207548364 wait complete. 2024-03-29T04:10:14.019 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:10:14.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.152+0000 7faae2bf0640 1 Processor -- start 2024-03-29T04:10:14.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.152+0000 7faae2bf0640 1 -- start start 2024-03-29T04:10:14.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae2bf0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faadc15ea70 0x7faadc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae2bf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 0x7faadc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae2bf0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faadc06e420 con 0x7faadc06d650 2024-03-29T04:10:14.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae2bf0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faadc072010 con 0x7faadc15ea70 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae2bf0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faadc071770 con 0x7faadc0cfd50 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faadbfff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faadc15ea70 0x7faadc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae098c640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7faadc06d650 0x7faadc06c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:56930/0 (socket says 172.21.15.181:56930) 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae118d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 0x7faadc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae098c640 1 -- 172.21.15.181:0/459916463 learned_addr learned my addr 172.21.15.181:0/459916463 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae118d640 1 -- 172.21.15.181:0/459916463 >> v2:172.21.15.187:3300/0 conn(0x7faadc15ea70 msgr2=0x7faadc0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae118d640 1 --2- 172.21.15.181:0/459916463 >> v2:172.21.15.187:3300/0 conn(0x7faadc15ea70 0x7faadc0710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae118d640 1 -- 172.21.15.181:0/459916463 >> v1:172.21.15.187:6789/0 conn(0x7faadc06d650 legacy=0x7faadc06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.153+0000 7faae118d640 1 -- 172.21.15.181:0/459916463 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc071a50 con 0x7faadc0cfd50 2024-03-29T04:10:14.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.154+0000 7faae118d640 1 --2- 172.21.15.181:0/459916463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 0x7faadc0d01e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d90 tx=0x7faad402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0697bc9f8d9ef21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.154+0000 7faadb7fe640 1 -- 172.21.15.181:0/459916463 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4039070 con 0x7faadc0cfd50 2024-03-29T04:10:14.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.154+0000 7faadb7fe640 1 -- 172.21.15.181:0/459916463 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad402cbc0 con 0x7faadc0cfd50 2024-03-29T04:10:14.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.154+0000 7faadb7fe640 1 -- 172.21.15.181:0/459916463 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4034440 con 0x7faadc0cfd50 2024-03-29T04:10:14.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 -- 172.21.15.181:0/459916463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 msgr2=0x7faadc0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 --2- 172.21.15.181:0/459916463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 0x7faadc0d01e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d90 tx=0x7faad402be10 comp rx=0 tx=0).stop 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 -- 172.21.15.181:0/459916463 shutdown_connections 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 --2- 172.21.15.181:0/459916463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc0cfd50 0x7faadc0d01e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 --2- 172.21.15.181:0/459916463 >> v2:172.21.15.187:3300/0 conn(0x7faadc15ea70 0x7faadc0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 -- 172.21.15.181:0/459916463 >> 172.21.15.181:0/459916463 conn(0x7faadc0d2180 msgr2=0x7faadc0d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 -- 172.21.15.181:0/459916463 shutdown_connections 2024-03-29T04:10:14.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.155+0000 7faae2bf0640 1 -- 172.21.15.181:0/459916463 wait complete. 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.156+0000 7faae2bf0640 1 Processor -- start 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.156+0000 7faae2bf0640 1 -- start start 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc06d650 0x7faadc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faadc15ea70 0x7faadc15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faadc06fed0 con 0x7faadc0cfd50 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faadc06f9b0 con 0x7faadc15ea70 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae2bf0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faadc070050 con 0x7faadc06d650 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faadbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faadbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34028/0 (socket says 172.21.15.181:34028) 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faadbfff640 1 -- 172.21.15.181:0/3595495682 learned_addr learned my addr 172.21.15.181:0/3595495682 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:14.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae118d640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faadc15ea70 0x7faadc15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.157+0000 7faae098c640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc06d650 0x7faadc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 -- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc06d650 msgr2=0x7faadc0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc06d650 0x7faadc0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 -- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faadc15ea70 msgr2=0x7faadc15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faadc15ea70 0x7faadc15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 -- 172.21.15.181:0/3595495682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc1f42d0 con 0x7faadc0cfd50 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faadbfff640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9b0 tx=0x7faac800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8017070 con 0x7faadc0cfd50 2024-03-29T04:10:14.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac800c9a0 con 0x7faadc0cfd50 2024-03-29T04:10:14.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc1f4570 con 0x7faadc0cfd50 2024-03-29T04:10:14.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac800cd90 con 0x7faadc0cfd50 2024-03-29T04:10:14.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.158+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1f4b30 con 0x7faadc0cfd50 2024-03-29T04:10:14.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.159+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc075900 con 0x7faadc0cfd50 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.159+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faac801a070 con 0x7faadc0cfd50 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.160+0000 7faad97fa640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 0x7faac006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.160+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7faac808bcd0 con 0x7faadc0cfd50 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.160+0000 7faae098c640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 0x7faac006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.160+0000 7faae098c640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 0x7faac006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faad0007fd0 tx=0x7faad0007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:14.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.164+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faac8059a10 con 0x7faadc0cfd50 2024-03-29T04:10:14.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:14.338+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faadc0b8100 con 0x7faadc0cfd50 2024-03-29T04:10:15.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.027+0000 7faad97fa640 1 -- 172.21.15.181:0/3595495682 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7faac805dd10 con 0x7faadc0cfd50 2024-03-29T04:10:15.007 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2024-03-29T04:10:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 msgr2=0x7faac006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 0x7faac006e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faad0007fd0 tx=0x7faad0007e80 comp rx=0 tx=0).stop 2024-03-29T04:10:15.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 msgr2=0x7faadc1f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9b0 tx=0x7faac800ee70 comp rx=0 tx=0).stop 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 shutdown_connections 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faadc15ea70 0x7faadc15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faadc0cfd50 0x7faadc1f7e10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faac006c530 0x7faac006e9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 --2- 172.21.15.181:0/3595495682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faadc06d650 0x7faadc0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 >> 172.21.15.181:0/3595495682 conn(0x7faadc0d2180 msgr2=0x7faadc074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 shutdown_connections 2024-03-29T04:10:15.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.030+0000 7faae2bf0640 1 -- 172.21.15.181:0/3595495682 wait complete. 2024-03-29T04:10:15.022 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-29T04:10:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.148+0000 7efe2c552640 1 Processor -- start 2024-03-29T04:10:15.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.148+0000 7efe2c552640 1 -- start start 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2c552640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe2406d930 0x7efe2415c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2c552640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 0x7efe240cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2c552640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe24160120 con 0x7efe2413f2d0 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2c552640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7efe2415d220 con 0x7efe2406d930 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2c552640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efe2415d470 con 0x7efe2415cdd0 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe2406d930 0x7efe2415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe2406d930 0x7efe2415c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34040/0 (socket says 172.21.15.181:34040) 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2a2ee640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efe2413f2d0 0x7efe2406d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48162/0 (socket says 172.21.15.181:48162) 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 -- 172.21.15.181:0/1357546811 learned_addr learned my addr 172.21.15.181:0/1357546811 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 -- 172.21.15.181:0/1357546811 >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 msgr2=0x7efe240cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2aaef640 1 --2- 172.21.15.181:0/1357546811 >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 0x7efe240cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 --2- 172.21.15.181:0/1357546811 >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 0x7efe240cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 -- 172.21.15.181:0/1357546811 >> v1:172.21.15.181:6789/0 conn(0x7efe2413f2d0 legacy=0x7efe2406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe292ec640 1 -- 172.21.15.181:0/1357546811 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836920563 0 0) 0x7efe24160120 con 0x7efe2413f2d0 2024-03-29T04:10:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe29aed640 1 -- 172.21.15.181:0/1357546811 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2415d750 con 0x7efe2406d930 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.149+0000 7efe2aaef640 1 --2- 172.21.15.181:0/1357546811 >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 0x7efe240cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.150+0000 7efe29aed640 1 --2- 172.21.15.181:0/1357546811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2406d930 0x7efe2415c890 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efe2406c430 tx=0x7efe1402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781d5fda408c481d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.150+0000 7efe292ec640 1 -- 172.21.15.181:0/1357546811 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14034070 con 0x7efe2406d930 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.150+0000 7efe292ec640 1 -- 172.21.15.181:0/1357546811 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe14030860 con 0x7efe2406d930 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.150+0000 7efe292ec640 1 -- 172.21.15.181:0/1357546811 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14030c30 con 0x7efe2406d930 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 -- 172.21.15.181:0/1357546811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2406d930 msgr2=0x7efe2415c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 --2- 172.21.15.181:0/1357546811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2406d930 0x7efe2415c890 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efe2406c430 tx=0x7efe1402b9a0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 -- 172.21.15.181:0/1357546811 shutdown_connections 2024-03-29T04:10:15.139 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 --2- 172.21.15.181:0/1357546811 >> v2:172.21.15.187:3300/0 conn(0x7efe2415cdd0 0x7efe240cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 --2- 172.21.15.181:0/1357546811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2406d930 0x7efe2415c890 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 -- 172.21.15.181:0/1357546811 >> 172.21.15.181:0/1357546811 conn(0x7efe2406f060 msgr2=0x7efe2406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 -- 172.21.15.181:0/1357546811 shutdown_connections 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.151+0000 7efe2c552640 1 -- 172.21.15.181:0/1357546811 wait complete. 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.152+0000 7efe2c552640 1 Processor -- start 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.152+0000 7efe2c552640 1 -- start start 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 0x7efe241524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe2413f2d0 0x7efe24158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe2413f6b0 con 0x7efe2415cdd0 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cf140 con 0x7efe2413f2d0 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2c552640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe240cea80 con 0x7efe2406d930 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34044/0 (socket says 172.21.15.181:34044) 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 -- 172.21.15.181:0/2167421389 learned_addr learned my addr 172.21.15.181:0/2167421389 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe29aed640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe2413f2d0 0x7efe24158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2a2ee640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 0x7efe241524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 -- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 msgr2=0x7efe241524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 0x7efe241524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 -- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe2413f2d0 msgr2=0x7efe24158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe2413f2d0 0x7efe24158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 -- 172.21.15.181:0/2167421389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe240b0880 con 0x7efe2415cdd0 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2a2ee640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 0x7efe241524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe2aaef640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c7c0 tx=0x7efe2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:15.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.153+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20015020 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.154+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe20004230 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.154+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe240b0b60 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.154+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20003610 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.154+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe240b1120 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.155+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe24075900 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.155+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efe20018020 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.155+0000 7efe1affd640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 0x7efe0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.155+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7efe2008b870 con 0x7efe2415cdd0 2024-03-29T04:10:15.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.155+0000 7efe2a2ee640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 0x7efe0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.156+0000 7efe2a2ee640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 0x7efe0806e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0012e0 tx=0x7efe1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:15.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.160+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efe2001f020 con 0x7efe2415cdd0 2024-03-29T04:10:15.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:15.333+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe2406c590 con 0x7efe2415cdd0 2024-03-29T04:10:16.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.032+0000 7efe1affd640 1 -- 172.21.15.181:0/2167421389 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7efe20059e30 con 0x7efe2415cdd0 2024-03-29T04:10:16.012 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data_ec' removed 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 msgr2=0x7efe0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 0x7efe0806e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0012e0 tx=0x7efe1c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 msgr2=0x7efe24158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c7c0 tx=0x7efe2000cc80 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 shutdown_connections 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe2415cdd0 0x7efe24158fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe2413f2d0 0x7efe24158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0806c530 0x7efe0806e9f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 --2- 172.21.15.181:0/2167421389 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe2406d930 0x7efe241524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 >> 172.21.15.181:0/2167421389 conn(0x7efe2406f060 msgr2=0x7efe2415bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 shutdown_connections 2024-03-29T04:10:16.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.035+0000 7efe2c552640 1 -- 172.21.15.181:0/2167421389 wait complete. 2024-03-29T04:10:16.030 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T04:10:16.030 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T04:10:16.031 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:10:16.031 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:16.031 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T04:10:16.031 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T04:10:16.031 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:10:16.031 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:16.031 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T04:10:16.032 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T04:10:16.032 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:10:16.032 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:16.032 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T04:10:16.032 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T04:10:16.032 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:10:16.032 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:10:16.043 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:10:16.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T04:10:16.045 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:10:16.045 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T04:10:16.045 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:10:16.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T04:10:16.046 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:10:16.046 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T04:10:16.087 INFO:tasks.ceph.mds.a:Started 2024-03-29T04:10:16.087 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:16.088 INFO:tasks.ceph.mds.b:Started 2024-03-29T04:10:16.088 INFO:tasks.ceph.mds.d:Started 2024-03-29T04:10:16.088 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:16.088 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:16.089 INFO:tasks.ceph.mds.c:Started 2024-03-29T04:10:16.089 DEBUG:teuthology.parallel:result is None 2024-03-29T04:10:16.089 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T04:10:16.167 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2024-03-29T04:10:16.168 INFO:tasks.ceph.mds.c.smithi181.stdout:starting mds.c at 2024-03-29T04:10:16.171 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-03-29T04:10:16.171 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-03-29T04:10:16.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.218+0000 7f8e83dd5640 1 Processor -- start 2024-03-29T04:10:16.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.218+0000 7f8e83dd5640 1 -- start start 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e83dd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e83dd5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e7c15f890 0x7f8e7c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e83dd5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e7c15feb0 con 0x7f8e7c0cbd00 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e83dd5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8e7c15c480 con 0x7f8e7c15f890 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e83dd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0cc140 con 0x7f8e7c0cb050 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e82372640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e7c15f890 0x7f8e7c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e82372640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8e7c15f890 0x7f8e7c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34076/0 (socket says 172.21.15.181:34076) 2024-03-29T04:10:16.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e82372640 1 -- 172.21.15.181:0/258453549 learned_addr learned my addr 172.21.15.181:0/258453549 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:16.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.219+0000 7f8e81b71640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e82372640 1 -- 172.21.15.181:0/258453549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 msgr2=0x7f8e7c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:16.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e82372640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e82372640 1 -- 172.21.15.181:0/258453549 >> v1:172.21.15.187:6789/0 conn(0x7f8e7c0cbd00 legacy=0x7f8e7c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.200 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e82372640 1 -- 172.21.15.181:0/258453549 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c15c910 con 0x7f8e7c15f890 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e82372640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c151c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e50 tx=0x7f8e7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9191a3d11aefae46 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.220+0000 7f8e80b6f640 1 -- 172.21.15.181:0/258453549 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7402cb00 con 0x7f8e7c15f890 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e80b6f640 1 -- 172.21.15.181:0/258453549 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7402cca0 con 0x7f8e7c15f890 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e80b6f640 1 -- 172.21.15.181:0/258453549 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74034610 con 0x7f8e7c15f890 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 -- 172.21.15.181:0/258453549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 msgr2=0x7f8e7c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c151c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e50 tx=0x7f8e7402bf60 comp rx=0 tx=0).stop 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 -- 172.21.15.181:0/258453549 shutdown_connections 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c151c20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/258453549 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 -- 172.21.15.181:0/258453549 >> 172.21.15.181:0/258453549 conn(0x7f8e7c06d610 msgr2=0x7f8e7c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:16.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 -- 172.21.15.181:0/258453549 shutdown_connections 2024-03-29T04:10:16.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.221+0000 7f8e83dd5640 1 -- 172.21.15.181:0/258453549 wait complete. 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.222+0000 7f8e83dd5640 1 Processor -- start 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.222+0000 7f8e83dd5640 1 -- start start 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e7c0cbd00 0x7f8e7c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c15c110 con 0x7f8e7c15f890 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c15be30 con 0x7f8e7c0cbd00 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e83dd5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0cb430 con 0x7f8e7c0cb050 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34092/0 (socket says 172.21.15.181:34092) 2024-03-29T04:10:16.204 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 -- 172.21.15.181:0/2929483184 learned_addr learned my addr 172.21.15.181:0/2929483184 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e81370640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e7c0cbd00 0x7f8e7c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e81b71640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 -- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 msgr2=0x7f8e7c0c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 -- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e7c0cbd00 msgr2=0x7f8e7c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e7c0cbd00 0x7f8e7c15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.223+0000 7f8e82372640 1 -- 172.21.15.181:0/2929483184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c1517a0 con 0x7f8e7c15f890 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e82372640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e50 tx=0x7f8e74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74043020 con 0x7f8e7c15f890 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7402ce50 con 0x7f8e7c15f890 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c0c66a0 con 0x7f8e7c15f890 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74009510 con 0x7f8e7c15f890 2024-03-29T04:10:16.205 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.224+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c15a150 con 0x7f8e7c15f890 2024-03-29T04:10:16.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.225+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c075900 con 0x7f8e7c15f890 2024-03-29T04:10:16.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.225+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8e74046020 con 0x7f8e7c15f890 2024-03-29T04:10:16.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.226+0000 7f8e7a7fc640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 0x7f8e6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:16.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.226+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f8e740ad9a0 con 0x7f8e7c15f890 2024-03-29T04:10:16.206 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.226+0000 7f8e81b71640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 0x7f8e6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:16.207 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.226+0000 7f8e81b71640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 0x7f8e6806e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009b40 tx=0x7f8e6c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:16.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.230+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8e7404d030 con 0x7f8e7c15f890 2024-03-29T04:10:16.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:16.416+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8e7c06f6b0 con 0x7f8e7c15f890 2024-03-29T04:10:17.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.042+0000 7f8e7a7fc640 1 -- 172.21.15.181:0/2929483184 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f8e7407bae0 con 0x7f8e7c15f890 2024-03-29T04:10:17.022 INFO:teuthology.orchestra.run.smithi181.stderr: removed all blocklist entries 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.044+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 msgr2=0x7f8e6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.044+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 0x7f8e6806e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009b40 tx=0x7f8e6c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.044+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 msgr2=0x7f8e7c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.044+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e50 tx=0x7f8e74004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 shutdown_connections 2024-03-29T04:10:17.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e7c15f890 0x7f8e7c0ccad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e7c0cbd00 0x7f8e7c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8e6806c530 0x7f8e6806e9f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 --2- 172.21.15.181:0/2929483184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8e7c0cb050 0x7f8e7c0c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 >> 172.21.15.181:0/2929483184 conn(0x7f8e7c06d610 msgr2=0x7f8e7c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 shutdown_connections 2024-03-29T04:10:17.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.045+0000 7f8e83dd5640 1 -- 172.21.15.181:0/2929483184 wait complete. 2024-03-29T04:10:17.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T04:10:17.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.158+0000 7f55da85b640 1 Processor -- start 2024-03-29T04:10:17.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.158+0000 7f55da85b640 1 -- start start 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55da85b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55da85b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d40724e0 con 0x7f55d40cb3b0 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55da85b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d40727c0 con 0x7f55d415a960 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55da85b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55d40730f0 con 0x7f55d40cd3b0 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55d8df8640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f55d40cb3b0 0x7f55d40cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:48176/0 (socket says 172.21.15.181:48176) 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55d8df8640 1 -- 172.21.15.181:0/1287717179 learned_addr learned my addr 172.21.15.181:0/1287717179 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55d37fe640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016502523 0 0) 0x7f55d40724e0 con 0x7f55d40cb3b0 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.159+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55bc003520 con 0x7f55d40cb3b0 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167951539 0 0) 0x7f55d40727c0 con 0x7f55d415a960 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55d40724e0 con 0x7f55d415a960 2024-03-29T04:10:17.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 70846310 0 0) 0x7f55bc003520 con 0x7f55d40cb3b0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 msgr2=0x7f55d40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 >> v1:172.21.15.187:6789/0 conn(0x7f55d415a960 legacy=0x7f55d4071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d4072aa0 con 0x7f55d40cb3b0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f55cc0019f0 con 0x7f55d40cb3b0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 >> v1:172.21.15.181:6789/0 conn(0x7f55d40cb3b0 legacy=0x7f55d40cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55bc003b70 0x7f55bc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc004d40 0x7f55bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40730f0 con 0x7f55bc004d40 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00ba70 con 0x7f55bc003b70 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc00bc70 con 0x7f55d40cd3b0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d8df8640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc004d40 0x7f55bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d3fff640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55bc003b70 0x7f55bc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 -- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55bc003b70 msgr2=0x7f55bc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55bc003b70 0x7f55bc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 -- 172.21.15.181:0/1287717179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc004d40 msgr2=0x7f55bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc004d40 0x7f55bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d37fe640 1 -- 172.21.15.181:0/1287717179 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc003520 con 0x7f55d40cd3b0 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.160+0000 7f55d8df8640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc004d40 0x7f55bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.161+0000 7f55d37fe640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55bc004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ac0 tx=0x7f55bc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d240d113f25461 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.161+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8039070 con 0x7f55d40cd3b0 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.161+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c802c710 con 0x7f55d40cd3b0 2024-03-29T04:10:17.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.161+0000 7f55d2ffd640 1 -- 172.21.15.181:0/1287717179 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8009690 con 0x7f55d40cd3b0 2024-03-29T04:10:17.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 -- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 msgr2=0x7f55bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 --2- 172.21.15.181:0/1287717179 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55bc004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ac0 tx=0x7f55bc005830 comp rx=0 tx=0).stop 2024-03-29T04:10:17.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55d8df8640 1 -- 172.21.15.181:0/1287717179 reap_dead start 2024-03-29T04:10:17.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 -- 172.21.15.181:0/1287717179 shutdown_connections 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 -- 172.21.15.181:0/1287717179 >> 172.21.15.181:0/1287717179 conn(0x7f55d40b1c90 msgr2=0x7f55d40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 -- 172.21.15.181:0/1287717179 shutdown_connections 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.162+0000 7f55da85b640 1 -- 172.21.15.181:0/1287717179 wait complete. 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.163+0000 7f55da85b640 1 Processor -- start 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.163+0000 7f55da85b640 1 -- start start 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d4152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d41587b0 0x7f55d414dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4072180 con 0x7f55d40c6800 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55d415ad40 con 0x7f55d41587b0 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55da85b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55d40cae70 con 0x7f55d40cd3b0 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:34102/0 (socket says 172.21.15.181:34102) 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 -- 172.21.15.181:0/764288929 learned_addr learned my addr 172.21.15.181:0/764288929 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:17.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d3fff640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d4152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d37fe640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d41587b0 0x7f55d414dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 -- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 msgr2=0x7f55d4152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d4152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 -- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d41587b0 msgr2=0x7f55d414dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d41587b0 0x7f55d414dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 -- 172.21.15.181:0/764288929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41594e0 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d8df8640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f55d40b1c20 tx=0x7f55cc00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.164+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00a330 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.165+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55cc018070 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.165+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d4158f20 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.165+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0133f0 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.165+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41566d0 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.166+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d4075a10 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.166+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f55cc013590 con 0x7f55d40c6800 2024-03-29T04:10:17.148 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.167+0000 7f55d0ff9640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 0x7f55c006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.167+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f55cc08b860 con 0x7f55d40c6800 2024-03-29T04:10:17.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.167+0000 7f55d3fff640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 0x7f55c006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.167+0000 7f55d3fff640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 0x7f55c006ea20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55c40019a0 tx=0x7f55c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.171+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f55cc0599f0 con 0x7f55d40c6800 2024-03-29T04:10:17.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.410+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f55d4074c90 con 0x7f55d40c6800 2024-03-29T04:10:17.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.411+0000 7f55d0ff9640 1 -- 172.21.15.181:0/764288929 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f55cc05dcf0 con 0x7f55d40c6800 2024-03-29T04:10:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:17.390 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:10:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "auth_dump": [ 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.a", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZm+TbyHhAAYS5vSs4/+lQVVlgLg3zhHw==", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.b", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZmEsdzKxAAW5MeNb4ibn6ssrofWq+OIA==", 2024-03-29T04:10:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.c", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZmu+FXJhAARrpjcV/v0swLv8eCgzyfMA==", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mds.d", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZmGlPPMhAAJAvKfZWz8U9XtO4+O8535Q==", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.0", 2024-03-29T04:10:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCwPgZmuIVAHBAAaGjP9Mlwc0CDVLVSXbAkwg==", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.1", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCzPgZmmX/GGhAAiihsU2WYAp46yWDHHzBfMA==", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.2", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC2PgZmxVUOGRAAE2idq11w4BUXWG60f96J4A==", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.3", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC5PgZmbn2HFxAAIjWELGbJT6yyL2rKOckWcA==", 2024-03-29T04:10:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.4", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQC/PgZmKV2GHRAAycPnjdnl/wtlL+R6zCX51Q==", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.5", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDCPgZmAY5PHBAAOG9GOPUe/fVQi6FfZg3tPQ==", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.6", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDFPgZmRdvwGhAAxpNEqLXzPuG9d6PVNBhJiw==", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "osd.7", 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDIPgZmnQN5GRAA3k64GtOezCHgN2msx/+jEg==", 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow profile osd", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile osd", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.0", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZm/aIPNxAAtuCXeNhwViJIHObDAGDzrw==", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2024-03-29T04:10:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.1", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtPgZm0P69BBAA6BlHVG2jLo5NyRVPct4M8Q==", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.2", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtPgZmPYBWARAATTtpn5FUlfeGNmSnUOKdVg==", 2024-03-29T04:10:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.3", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCtPgZm1Zt/ChAArZhRTQqCS4S1dBOBh+imWw==", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow r", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow rw", 2024-03-29T04:10:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow rwx" 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.admin", 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZmZg6aARAAEa7D5qIxfOVfbE8rT2rczQ==", 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "mgr": "allow *", 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow *", 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mds", 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZm2/mSFhAAdNY4tp9T2GQlUf/IlJMRiQ==", 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZm1wKTFhAAhB93+Nyf2S0QvE3PSqGr/w==", 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-osd", 2024-03-29T04:10:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZm7wuTFhAAB3uBKN0yJwxlOMF7nkDigQ==", 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZm4hOTFhAAsZ6bEAcBeXp0NZyG7QqIfg==", 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZm4huTFhAAW6cSRWgyq67mmUiE/dM9AA==", 2024-03-29T04:10:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQDTPgZmRiOTFhAAuSYdJks6yVHtJmbpXVvxUw==", 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.x", 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZmjT+EGxAAv1PsM4wkpglDq1aHTMHodQ==", 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "entity": "mgr.y", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "AQCsPgZm7VLeFhAAxF0G6FHc+xEuTPHFnIx85A==", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": { 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": "allow *", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "mon": "allow profile mgr", 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "osd": "allow *" 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-03-29T04:10:17.407 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 msgr2=0x7f55c006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 0x7f55c006ea20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55c40019a0 tx=0x7f55c4009040 comp rx=0 tx=0).stop 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 msgr2=0x7f55d40c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f55d40b1c20 tx=0x7f55cc00af70 comp rx=0 tx=0).stop 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 shutdown_connections 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d40c6800 0x7f55d40c6c80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d41587b0 0x7f55d414dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c006c560 0x7f55c006ea20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 --2- 172.21.15.181:0/764288929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d40cd3b0 0x7f55d4152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 >> 172.21.15.181:0/764288929 conn(0x7f55d40b1c90 msgr2=0x7f55d40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 shutdown_connections 2024-03-29T04:10:17.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.413+0000 7f55da85b640 1 -- 172.21.15.181:0/764288929 wait complete. 2024-03-29T04:10:17.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T04:10:17.512 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.532+0000 7fcbf99ef640 1 Processor -- start 2024-03-29T04:10:17.512 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.532+0000 7fcbf99ef640 1 -- start start 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf99ef640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbf415fe10 0x7fcbf40b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf99ef640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf40cced0 con 0x7fcbf415c260 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf99ef640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf4071110 con 0x7fcbf40b4130 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf99ef640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf40713f0 con 0x7fcbf415fe10 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf27fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbf415fe10 0x7fcbf40b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf2ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fcbf415c260 0x7fcbf40cc7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54476/0 (socket says 172.21.15.181:54476) 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.533+0000 7fcbf2ffd640 1 -- 172.21.15.181:0/2786885948 learned_addr learned my addr 172.21.15.181:0/2786885948 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:17.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf27fc640 1 -- 172.21.15.181:0/2786885948 >> v1:172.21.15.187:6789/0 conn(0x7fcbf40b4130 legacy=0x7fcbf415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.514 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf27fc640 1 -- 172.21.15.181:0/2786885948 >> v1:172.21.15.181:6789/0 conn(0x7fcbf415c260 legacy=0x7fcbf40cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.514 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf27fc640 1 -- 172.21.15.181:0/2786885948 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf40716d0 con 0x7fcbf415fe10 2024-03-29T04:10:17.514 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf1ffb640 1 -- 172.21.15.181:0/2786885948 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691653450 0 0) 0x7fcbf40cced0 con 0x7fcbf415c260 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf27fc640 1 --2- 172.21.15.181:0/2786885948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415fe10 0x7fcbf40b3bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80097a0 tx=0x7fcbe802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ddac6f5954b0f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf1ffb640 1 -- 172.21.15.181:0/2786885948 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8035020 con 0x7fcbf415fe10 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf1ffb640 1 -- 172.21.15.181:0/2786885948 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe8031830 con 0x7fcbf415fe10 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.534+0000 7fcbf1ffb640 1 -- 172.21.15.181:0/2786885948 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8031c00 con 0x7fcbf415fe10 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 -- 172.21.15.181:0/2786885948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415fe10 msgr2=0x7fcbf40b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/2786885948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415fe10 0x7fcbf40b3bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80097a0 tx=0x7fcbe802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 -- 172.21.15.181:0/2786885948 shutdown_connections 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/2786885948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf415fe10 0x7fcbf40b3bf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 -- 172.21.15.181:0/2786885948 >> 172.21.15.181:0/2786885948 conn(0x7fcbf40cebf0 msgr2=0x7fcbf40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.515 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 -- 172.21.15.181:0/2786885948 shutdown_connections 2024-03-29T04:10:17.516 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.535+0000 7fcbf99ef640 1 -- 172.21.15.181:0/2786885948 wait complete. 2024-03-29T04:10:17.517 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.536+0000 7fcbf99ef640 1 Processor -- start 2024-03-29T04:10:17.517 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.536+0000 7fcbf99ef640 1 -- start start 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf415c260 0x7fcbf40cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf415fe10 0x7fcbf40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf415c640 con 0x7fcbf40b4130 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4071c90 con 0x7fcbf415c260 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf99ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf40ce640 con 0x7fcbf415fe10 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54772/0 (socket says 172.21.15.181:54772) 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 -- 172.21.15.181:0/794812400 learned_addr learned my addr 172.21.15.181:0/794812400 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 -- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf415fe10 msgr2=0x7fcbf40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf415fe10 0x7fcbf40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 -- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf415c260 msgr2=0x7fcbf40cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf415c260 0x7fcbf40cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.518 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.537+0000 7fcbf2ffd640 1 -- 172.21.15.181:0/794812400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf415b800 con 0x7fcbf40b4130 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbf2ffd640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400c8c0 tx=0x7fcbe400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4018070 con 0x7fcbf40b4130 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe400a640 con 0x7fcbf40b4130 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf40cc770 con 0x7fcbf40b4130 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400aa30 con 0x7fcbf40b4130 2024-03-29T04:10:17.519 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.538+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf40c9630 con 0x7fcbf40b4130 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.539+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf4075900 con 0x7fcbf40b4130 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.539+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcbe401b070 con 0x7fcbf40b4130 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.539+0000 7fcbdbfff640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 0x7fcbe006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.540+0000 7fcbf27fc640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 0x7fcbe006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.540+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fcbe408ab90 con 0x7fcbf40b4130 2024-03-29T04:10:17.520 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.540+0000 7fcbf27fc640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 0x7fcbe006e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8036000 tx=0x7fcbe8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.523 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.544+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcbf40cc770 con 0x7fcbf40b4130 2024-03-29T04:10:17.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.786+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fcbf40702d0 con 0x7fcbf40b4130 2024-03-29T04:10:17.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.788+0000 7fcbdbfff640 1 -- 172.21.15.181:0/794812400 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fcbe405f030 con 0x7fcbf40b4130 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 msgr2=0x7fcbe006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 0x7fcbe006e9f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8036000 tx=0x7fcbe8030000 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 msgr2=0x7fcbf415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400c8c0 tx=0x7fcbe400cd80 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 shutdown_connections 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbe006c530 0x7fcbe006e9f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbf415fe10 0x7fcbf40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbf415c260 0x7fcbf40cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 --2- 172.21.15.181:0/794812400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf40b4130 0x7fcbf415b2c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 >> 172.21.15.181:0/794812400 conn(0x7fcbf40cebf0 msgr2=0x7fcbf415e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 shutdown_connections 2024-03-29T04:10:17.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.791+0000 7fcbf99ef640 1 -- 172.21.15.181:0/794812400 wait complete. 2024-03-29T04:10:17.781 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T04:10:17.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.903+0000 7f886b855640 1 Processor -- start 2024-03-29T04:10:17.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 -- start start 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 0x7f88640cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f88640cec60 0x7f88640d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88640d3ee0 0x7f88640d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f88640cfc40 con 0x7f88640d3ee0 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f88640d3120 con 0x7f88640cec60 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.904+0000 7f886b855640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88640d3400 con 0x7f886415e7a0 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88640d3ee0 0x7f88640d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88640d3ee0 0x7f88640d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54786/0 (socket says 172.21.15.181:54786) 2024-03-29T04:10:17.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 -- 172.21.15.181:0/10074330 learned_addr learned my addr 172.21.15.181:0/10074330 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f88695f1640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 0x7f88640cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8868df0640 1 --2- 172.21.15.181:0/10074330 >> v2:172.21.15.187:3300/0 conn(0x7f88640cec60 0x7f88640d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 -- 172.21.15.181:0/10074330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 msgr2=0x7f88640cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 0x7f88640cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 -- 172.21.15.181:0/10074330 >> v2:172.21.15.187:3300/0 conn(0x7f88640cec60 msgr2=0x7f88640d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 --2- 172.21.15.181:0/10074330 >> v2:172.21.15.187:3300/0 conn(0x7f88640cec60 0x7f88640d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f8869df2640 1 -- 172.21.15.181:0/10074330 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88640d36e0 con 0x7f88640d3ee0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.905+0000 7f88695f1640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 0x7f88640cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f8869df2640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640d3ee0 0x7f88640d17f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f88640d5490 tx=0x7f885c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c506ef7c0091086f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f885bfff640 1 -- 172.21.15.181:0/10074330 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c02c9e0 con 0x7f88640d3ee0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f885bfff640 1 -- 172.21.15.181:0/10074330 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c02cb80 con 0x7f88640d3ee0 2024-03-29T04:10:17.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f885bfff640 1 -- 172.21.15.181:0/10074330 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c0343f0 con 0x7f88640d3ee0 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 -- 172.21.15.181:0/10074330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640d3ee0 msgr2=0x7f88640d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640d3ee0 0x7f88640d17f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f88640d5490 tx=0x7f885c02be40 comp rx=0 tx=0).stop 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 -- 172.21.15.181:0/10074330 shutdown_connections 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640d3ee0 0x7f88640d17f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 --2- 172.21.15.181:0/10074330 >> v2:172.21.15.187:3300/0 conn(0x7f88640cec60 0x7f88640d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 --2- 172.21.15.181:0/10074330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f886415e7a0 0x7f88640cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 -- 172.21.15.181:0/10074330 >> 172.21.15.181:0/10074330 conn(0x7f88640676b0 msgr2=0x7f8864067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 -- 172.21.15.181:0/10074330 shutdown_connections 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.906+0000 7f886b855640 1 -- 172.21.15.181:0/10074330 wait complete. 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.907+0000 7f886b855640 1 Processor -- start 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 -- start start 2024-03-29T04:10:17.889 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88640d3ee0 0x7f886415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88640cd270 0x7f88640cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f88640b5e10 con 0x7f88640cec60 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88640cf930 con 0x7f88640d3ee0 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f886b855640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88640b5f90 con 0x7f88640cd270 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f88695f1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f88695f1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54788/0 (socket says 172.21.15.181:54788) 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.908+0000 7f88695f1640 1 -- 172.21.15.181:0/1690229793 learned_addr learned my addr 172.21.15.181:0/1690229793 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 -- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88640cd270 msgr2=0x7f88640cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88640cd270 0x7f88640cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 -- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88640d3ee0 msgr2=0x7f886415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88640d3ee0 0x7f886415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 -- 172.21.15.181:0/1690229793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88640cb6b0 con 0x7f88640cec60 2024-03-29T04:10:17.890 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f88695f1640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f885400c7f0 tx=0x7f885400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854015070 con 0x7f88640cec60 2024-03-29T04:10:17.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8854007820 con 0x7f88640cec60 2024-03-29T04:10:17.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88640cb990 con 0x7f88640cec60 2024-03-29T04:10:17.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854007bd0 con 0x7f88640cec60 2024-03-29T04:10:17.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.909+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88640cbf50 con 0x7f88640cec60 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.910+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88640cb440 con 0x7f88640cec60 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.910+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8854018070 con 0x7f88640cec60 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.911+0000 7f8859ffb640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 0x7f884806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.911+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f885408b550 con 0x7f88640cec60 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.911+0000 7f8868df0640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 0x7f884806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:17.892 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.911+0000 7f8868df0640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 0x7f884806e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:17.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:17.915+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8854059d30 con 0x7f88640cec60 2024-03-29T04:10:18.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.158+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f886415df30 con 0x7f88640cec60 2024-03-29T04:10:18.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.160+0000 7f8859ffb640 1 -- 172.21.15.181:0/1690229793 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f8854059630 con 0x7f88640cec60 2024-03-29T04:10:18.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.162+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 msgr2=0x7f884806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.162+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 0x7f884806e9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860008040 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 msgr2=0x7f88640cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f885400c7f0 tx=0x7f885400ccb0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 shutdown_connections 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f884806c530 0x7f884806e9f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88640cd270 0x7f88640cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88640d3ee0 0x7f886415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 --2- 172.21.15.181:0/1690229793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88640cec60 0x7f88640cc750 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 >> 172.21.15.181:0/1690229793 conn(0x7f88640676b0 msgr2=0x7f88640b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 shutdown_connections 2024-03-29T04:10:18.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.163+0000 7f886b855640 1 -- 172.21.15.181:0/1690229793 wait complete. 2024-03-29T04:10:18.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T04:10:18.254 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.275+0000 7efe19d82640 1 Processor -- start 2024-03-29T04:10:18.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.275+0000 7efe19d82640 1 -- start start 2024-03-29T04:10:18.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe140cb050 0x7efe140cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efe140cbd00 0x7efe1415f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7efe140cc170 con 0x7efe140cb050 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efe1415c480 con 0x7efe140cbd00 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe19d82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe1415feb0 con 0x7efe1415f6c0 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe140cb050 0x7efe140cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe140cb050 0x7efe140cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54804/0 (socket says 172.21.15.181:54804) 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 -- 172.21.15.181:0/4123956750 learned_addr learned my addr 172.21.15.181:0/4123956750 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe13fff640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe12ffd640 1 --2- 172.21.15.181:0/4123956750 >> v2:172.21.15.187:3300/0 conn(0x7efe140cbd00 0x7efe1415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 -- 172.21.15.181:0/4123956750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 msgr2=0x7efe14151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 -- 172.21.15.181:0/4123956750 >> v2:172.21.15.187:3300/0 conn(0x7efe140cbd00 msgr2=0x7efe1415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 --2- 172.21.15.181:0/4123956750 >> v2:172.21.15.187:3300/0 conn(0x7efe140cbd00 0x7efe1415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.276+0000 7efe137fe640 1 -- 172.21.15.181:0/4123956750 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1415c910 con 0x7efe140cb050 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe137fe640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cb050 0x7efe140cb7c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efe140c3b10 tx=0x7efe0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f738d3f9ae874 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.257 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe127fc640 1 -- 172.21.15.181:0/4123956750 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035020 con 0x7efe140cb050 2024-03-29T04:10:18.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe127fc640 1 -- 172.21.15.181:0/4123956750 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe04031870 con 0x7efe140cb050 2024-03-29T04:10:18.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe127fc640 1 -- 172.21.15.181:0/4123956750 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04031c40 con 0x7efe140cb050 2024-03-29T04:10:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe19d82640 1 -- 172.21.15.181:0/4123956750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cb050 msgr2=0x7efe140cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe19d82640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cb050 0x7efe140cb7c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efe140c3b10 tx=0x7efe0402b910 comp rx=0 tx=0).stop 2024-03-29T04:10:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.277+0000 7efe19d82640 1 -- 172.21.15.181:0/4123956750 shutdown_connections 2024-03-29T04:10:18.260 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 --2- 172.21.15.181:0/4123956750 >> v2:172.21.15.187:3300/0 conn(0x7efe140cbd00 0x7efe1415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 --2- 172.21.15.181:0/4123956750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cb050 0x7efe140cb7c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 -- 172.21.15.181:0/4123956750 >> 172.21.15.181:0/4123956750 conn(0x7efe1406d610 msgr2=0x7efe1406da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 -- 172.21.15.181:0/4123956750 shutdown_connections 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.278+0000 7efe19d82640 1 -- 172.21.15.181:0/4123956750 wait complete. 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.279+0000 7efe19d82640 1 Processor -- start 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.279+0000 7efe19d82640 1 -- start start 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe140cb050 0x7efe140b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe1415c110 con 0x7efe140cbd00 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe1415be30 con 0x7efe140cb050 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe1415fb50 con 0x7efe1415f6c0 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54818/0 (socket says 172.21.15.181:54818) 2024-03-29T04:10:18.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 -- 172.21.15.181:0/3200767581 learned_addr learned my addr 172.21.15.181:0/3200767581 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe13fff640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe137fe640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe140cb050 0x7efe140b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 -- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 msgr2=0x7efe14157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 -- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe140cb050 msgr2=0x7efe140b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe140cb050 0x7efe140b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 -- 172.21.15.181:0/3200767581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe141586d0 con 0x7efe140cbd00 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe12ffd640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d830 tx=0x7efe0800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08018020 con 0x7efe140cbd00 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe080096a0 con 0x7efe140cbd00 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08009a70 con 0x7efe140cbd00 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe141589b0 con 0x7efe140cbd00 2024-03-29T04:10:18.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.280+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141564b0 con 0x7efe140cbd00 2024-03-29T04:10:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.281+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe140cb430 con 0x7efe140cbd00 2024-03-29T04:10:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.282+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efe0801b020 con 0x7efe140cbd00 2024-03-29T04:10:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.282+0000 7efdfbfff640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 0x7efe0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.282+0000 7efe137fe640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 0x7efe0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.263 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.282+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7efe0808bad0 con 0x7efe140cbd00 2024-03-29T04:10:18.264 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.283+0000 7efe137fe640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 0x7efe0006e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efe04036000 tx=0x7efe0403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.287+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efe140cb430 con 0x7efe140cbd00 2024-03-29T04:10:18.508 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.528+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7efe14075900 con 0x7efe140cbd00 2024-03-29T04:10:18.510 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.530+0000 7efdfbfff640 1 -- 172.21.15.181:0/3200767581 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7efe08022020 con 0x7efe140cbd00 2024-03-29T04:10:18.512 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 msgr2=0x7efe0006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.512 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 0x7efe0006e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efe04036000 tx=0x7efe0403c040 comp rx=0 tx=0).stop 2024-03-29T04:10:18.512 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 msgr2=0x7efe1414a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d830 tx=0x7efe0800dcf0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 shutdown_connections 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efe0006c530 0x7efe0006e9f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe1415f6c0 0x7efe14157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe140cbd00 0x7efe1414a720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 --2- 172.21.15.181:0/3200767581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe140cb050 0x7efe140b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 >> 172.21.15.181:0/3200767581 conn(0x7efe1406d610 msgr2=0x7efe1406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 shutdown_connections 2024-03-29T04:10:18.513 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.533+0000 7efe19d82640 1 -- 172.21.15.181:0/3200767581 wait complete. 2024-03-29T04:10:18.523 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T04:10:18.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.645+0000 7f8aa3bdb640 1 Processor -- start 2024-03-29T04:10:18.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.645+0000 7f8aa3bdb640 1 -- start start 2024-03-29T04:10:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa3bdb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a9c15e5c0 0x7f8a9c072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa3bdb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a9c06bc90 0x7f8a9c06fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa3bdb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9c0b6540 con 0x7f8a9c0ce3e0 2024-03-29T04:10:18.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa3bdb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c06fb60 con 0x7f8a9c06bc90 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa3bdb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c0d0610 con 0x7f8a9c15e5c0 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a9c06bc90 0x7f8a9c06fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8a9c06bc90 0x7f8a9c06fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54824/0 (socket says 172.21.15.181:54824) 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 -- 172.21.15.181:0/3075268828 learned_addr learned my addr 172.21.15.181:0/3075268828 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 -- 172.21.15.181:0/3075268828 >> v2:172.21.15.187:3300/0 conn(0x7f8a9c15e5c0 msgr2=0x7f8a9c072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa1977640 1 --2- 172.21.15.181:0/3075268828 >> v2:172.21.15.187:3300/0 conn(0x7f8a9c15e5c0 0x7f8a9c072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 --2- 172.21.15.181:0/3075268828 >> v2:172.21.15.187:3300/0 conn(0x7f8a9c15e5c0 0x7f8a9c072590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 -- 172.21.15.181:0/3075268828 >> v1:172.21.15.187:6789/0 conn(0x7f8a9c0ce3e0 legacy=0x7f8a9c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.646+0000 7f8aa2178640 1 -- 172.21.15.181:0/3075268828 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c0d08f0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa2178640 1 --2- 172.21.15.181:0/3075268828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c06fff0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c069170 tx=0x7f8a9402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e0a863cb502c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa0975640 1 -- 172.21.15.181:0/3075268828 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402c9c0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa0975640 1 -- 172.21.15.181:0/3075268828 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9402cb60 con 0x7f8a9c06bc90 2024-03-29T04:10:18.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa0975640 1 -- 172.21.15.181:0/3075268828 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a940343f0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/3075268828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 msgr2=0x7f8a9c06fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.631 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.647+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/3075268828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c06fff0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c069170 tx=0x7f8a9402be20 comp rx=0 tx=0).stop 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/3075268828 shutdown_connections 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/3075268828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c06fff0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/3075268828 >> v2:172.21.15.187:3300/0 conn(0x7f8a9c15e5c0 0x7f8a9c072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/3075268828 >> 172.21.15.181:0/3075268828 conn(0x7f8a9c0d10f0 msgr2=0x7f8a9c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/3075268828 shutdown_connections 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.648+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/3075268828 wait complete. 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.649+0000 7f8aa3bdb640 1 Processor -- start 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.649+0000 7f8aa3bdb640 1 -- start start 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.649+0000 7f8aa3bdb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.649+0000 7f8aa3bdb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a9c0ce3e0 0x7f8a9c06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.649+0000 7f8aa3bdb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a9c15e5c0 0x7f8a9c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa3bdb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c15e9a0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa3bdb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c0d0eb0 con 0x7f8a9c0ce3e0 2024-03-29T04:10:18.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa3bdb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0ce870 con 0x7f8a9c15e5c0 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54828/0 (socket says 172.21.15.181:54828) 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 -- 172.21.15.181:0/2375270628 learned_addr learned my addr 172.21.15.181:0/2375270628 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa2178640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a9c15e5c0 0x7f8a9c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1176640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a9c0ce3e0 0x7f8a9c06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 -- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a9c15e5c0 msgr2=0x7f8a9c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a9c15e5c0 0x7f8a9c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 -- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a9c0ce3e0 msgr2=0x7f8a9c06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a9c0ce3e0 0x7f8a9c06e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 -- 172.21.15.181:0/2375270628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c14d780 con 0x7f8a9c06bc90 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa1977640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c820 tx=0x7f8a9000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90015070 con 0x7f8a9c06bc90 2024-03-29T04:10:18.633 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a90004240 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000ce10 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c1495f0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.650+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c149e50 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.651+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c1497c0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.652+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8a90007830 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.652+0000 7f8a8e7fc640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 0x7f8a8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.652+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f8a9008b360 con 0x7f8a9c06bc90 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.652+0000 7f8aa1176640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 0x7f8a8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:18.634 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.652+0000 7f8aa1176640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 0x7f8a8006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0b5640 tx=0x7f8a98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:18.635 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.656+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8a9c1497c0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.902+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f8a9c075900 con 0x7f8a9c06bc90 2024-03-29T04:10:18.883 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.904+0000 7f8a8e7fc640 1 -- 172.21.15.181:0/2375270628 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f8a90059bd0 con 0x7f8a9c06bc90 2024-03-29T04:10:18.885 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 msgr2=0x7f8a8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 0x7f8a8006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0b5640 tx=0x7f8a98009040 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 msgr2=0x7f8a9c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c820 tx=0x7f8a9000cce0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 shutdown_connections 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8a8006c530 0x7f8a8006e9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a9c15e5c0 0x7f8a9c06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a9c0ce3e0 0x7f8a9c06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 --2- 172.21.15.181:0/2375270628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a9c06bc90 0x7f8a9c0b22e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 >> 172.21.15.181:0/2375270628 conn(0x7f8a9c0d10f0 msgr2=0x7f8a9c072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 shutdown_connections 2024-03-29T04:10:18.886 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:18.906+0000 7f8aa3bdb640 1 -- 172.21.15.181:0/2375270628 wait complete. 2024-03-29T04:10:18.896 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T04:10:18.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.018+0000 7f953c075640 1 Processor -- start 2024-03-29T04:10:18.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.018+0000 7f953c075640 1 -- start start 2024-03-29T04:10:19.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953c075640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9534162030 0x7f95341624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953c075640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95340b5240 0x7f95340b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953c075640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9534141910 con 0x7f9534141530 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953c075640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b5bf0 con 0x7f95340b5240 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953c075640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9534162af0 con 0x7f9534162030 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f953a612640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95340b5240 0x7f95340b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f9539e11640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9534141530 0x7f9534071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54480/0 (socket says 172.21.15.181:54480) 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.019+0000 7f9539e11640 1 -- 172.21.15.181:0/3925272592 learned_addr learned my addr 172.21.15.181:0/3925272592 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f9539610640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9534162030 0x7f95341624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f953a612640 1 -- 172.21.15.181:0/3925272592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9534162030 msgr2=0x7f95341624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f953a612640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9534162030 0x7f95341624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f953a612640 1 -- 172.21.15.181:0/3925272592 >> v1:172.21.15.181:6789/0 conn(0x7f9534141530 legacy=0x7f9534071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f953a612640 1 -- 172.21.15.181:0/3925272592 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9534162dd0 con 0x7f95340b5240 2024-03-29T04:10:19.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f9538e0f640 1 -- 172.21.15.181:0/3925272592 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642995497 0 0) 0x7f9534141910 con 0x7f9534141530 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.020+0000 7f953a612640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95340b5240 0x7f95340b56b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f952c0097a0 tx=0x7f952c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32a304580e8dd45 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f9538e0f640 1 -- 172.21.15.181:0/3925272592 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c0315a0 con 0x7f95340b5240 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f9538e0f640 1 -- 172.21.15.181:0/3925272592 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c037070 con 0x7f95340b5240 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f9538e0f640 1 -- 172.21.15.181:0/3925272592 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c031960 con 0x7f95340b5240 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 -- 172.21.15.181:0/3925272592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95340b5240 msgr2=0x7f95340b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95340b5240 0x7f95340b56b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f952c0097a0 tx=0x7f952c02b840 comp rx=0 tx=0).stop 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 -- 172.21.15.181:0/3925272592 shutdown_connections 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95340b5240 0x7f95340b56b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 --2- 172.21.15.181:0/3925272592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9534162030 0x7f95341624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 -- 172.21.15.181:0/3925272592 >> 172.21.15.181:0/3925272592 conn(0x7f953406f8b0 msgr2=0x7f953406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 -- 172.21.15.181:0/3925272592 shutdown_connections 2024-03-29T04:10:19.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.021+0000 7f953c075640 1 -- 172.21.15.181:0/3925272592 wait complete. 2024-03-29T04:10:19.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.022+0000 7f953c075640 1 Processor -- start 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.022+0000 7f953c075640 1 -- start start 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95340b5240 0x7f9534156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9534141530 0x7f9534143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95340b1950 con 0x7f9534162030 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9534071400 con 0x7f9534141530 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953c075640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9534141c10 con 0x7f95340b5240 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953a612640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953a612640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54854/0 (socket says 172.21.15.181:54854) 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f953a612640 1 -- 172.21.15.181:0/3017792309 learned_addr learned my addr 172.21.15.181:0/3017792309 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f9539e11640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95340b5240 0x7f9534156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.023+0000 7f9539610640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9534141530 0x7f9534143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 -- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95340b5240 msgr2=0x7f9534156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95340b5240 0x7f9534156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 -- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9534141530 msgr2=0x7f9534143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9534141530 0x7f9534143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 -- 172.21.15.181:0/3017792309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341e8d00 con 0x7f9534162030 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953a612640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f952c034040 tx=0x7f952c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c004230 con 0x7f9534162030 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c0043d0 con 0x7f9534162030 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340b1df0 con 0x7f9534162030 2024-03-29T04:10:19.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c003850 con 0x7f9534162030 2024-03-29T04:10:19.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.024+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95340b23b0 con 0x7f9534162030 2024-03-29T04:10:19.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.025+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534075a10 con 0x7f9534162030 2024-03-29T04:10:19.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.025+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f952c0039f0 con 0x7f9534162030 2024-03-29T04:10:19.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.026+0000 7f952a7fc640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 0x7f951806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.026+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f952c02f030 con 0x7f9534162030 2024-03-29T04:10:19.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.026+0000 7f9539e11640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 0x7f951806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.026+0000 7f9539e11640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 0x7f951806e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f95300012e0 tx=0x7f9530001700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.030+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f952c041070 con 0x7f9534162030 2024-03-29T04:10:19.253 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.274+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f95341583c0 con 0x7f9534162030 2024-03-29T04:10:19.255 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.276+0000 7f952a7fc640 1 -- 172.21.15.181:0/3017792309 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f952c07b4c0 con 0x7f9534162030 2024-03-29T04:10:19.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 msgr2=0x7f951806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 0x7f951806e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f95300012e0 tx=0x7f9530001700 comp rx=0 tx=0).stop 2024-03-29T04:10:19.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 msgr2=0x7f9534143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f952c034040 tx=0x7f952c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 shutdown_connections 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9534162030 0x7f9534143560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9534141530 0x7f9534143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f951806c530 0x7f951806e9f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 --2- 172.21.15.181:0/3017792309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95340b5240 0x7f9534156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.278+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 >> 172.21.15.181:0/3017792309 conn(0x7f953406f8b0 msgr2=0x7f9534160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.279+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 shutdown_connections 2024-03-29T04:10:19.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.279+0000 7f953c075640 1 -- 172.21.15.181:0/3017792309 wait complete. 2024-03-29T04:10:19.268 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T04:10:19.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.390+0000 7f083daf1640 1 Processor -- start 2024-03-29T04:10:19.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.390+0000 7f083daf1640 1 -- start start 2024-03-29T04:10:19.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f083daf1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08380727f0 0x7f08380afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f083daf1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08380cb6d0 con 0x7f083806c590 2024-03-29T04:10:19.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f083daf1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083815e8e0 con 0x7f083815e500 2024-03-29T04:10:19.373 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f083daf1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f08380cc580 con 0x7f08380727f0 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f0837fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f083806c590 0x7f08380c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54496/0 (socket says 172.21.15.181:54496) 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f0837fff640 1 -- 172.21.15.181:0/319112351 learned_addr learned my addr 172.21.15.181:0/319112351 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644134911 0 0) 0x7f08380cb6d0 con 0x7f083806c590 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0820003520 con 0x7f083806c590 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f0836ffd640 1 --2- 172.21.15.181:0/319112351 >> v2:172.21.15.187:3300/0 conn(0x7f08380727f0 0x7f08380afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.391+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1805720333 0 0) 0x7f0820003520 con 0x7f083806c590 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 >> v2:172.21.15.187:3300/0 conn(0x7f08380727f0 msgr2=0x7f08380afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 --2- 172.21.15.181:0/319112351 >> v2:172.21.15.187:3300/0 conn(0x7f08380727f0 0x7f08380afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 >> v1:172.21.15.187:6789/0 conn(0x7f083815e500 legacy=0x7f08380cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380cb970 con 0x7f083806c590 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f08300019f0 con 0x7f083806c590 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 >> v1:172.21.15.181:6789/0 conn(0x7f083806c590 legacy=0x7f08380c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 0x7f0820003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820004800 0x7f0820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08200052e0 0x7f082000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08380cc580 con 0x7f08200039b0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f082000c1b0 con 0x7f08200052e0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f082000c3b0 con 0x7f0820004800 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 0x7f0820003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f0836ffd640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820004800 0x7f0820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f0837fff640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08200052e0 0x7f082000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 -- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820004800 msgr2=0x7f0820004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820004800 0x7f0820004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 -- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08200052e0 msgr2=0x7f082000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08200052e0 0x7f082000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f08377fe640 1 -- 172.21.15.181:0/319112351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0820003520 con 0x7f08200039b0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.392+0000 7f0837fff640 1 -- 172.21.15.181:0/319112351 reap_dead start 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f08377fe640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 0x7f0820003df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0828001750 tx=0x7f0820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7aaadceae8409d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828001370 con 0x7f08200039b0 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08280052d0 con 0x7f08200039b0 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f08367fc640 1 -- 172.21.15.181:0/319112351 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828003830 con 0x7f08200039b0 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 -- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 msgr2=0x7f0820003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 0x7f0820003df0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0828001750 tx=0x7f0820005830 comp rx=0 tx=0).stop 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 -- 172.21.15.181:0/319112351 shutdown_connections 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 --2- 172.21.15.181:0/319112351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200039b0 0x7f0820003df0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 -- 172.21.15.181:0/319112351 >> 172.21.15.181:0/319112351 conn(0x7f0838071d00 msgr2=0x7f08380720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 -- 172.21.15.181:0/319112351 shutdown_connections 2024-03-29T04:10:19.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.393+0000 7f083daf1640 1 -- 172.21.15.181:0/319112351 wait complete. 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.395+0000 7f083daf1640 1 Processor -- start 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.395+0000 7f083daf1640 1 -- start start 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.395+0000 7f083daf1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 0x7f0838152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.395+0000 7f083daf1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 0x7f083815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08380cc210 con 0x7f083806f0d0 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08380cbf30 con 0x7f083815aae0 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08380afcb0 con 0x7f08200039b0 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54876/0 (socket says 172.21.15.181:54876) 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 -- 172.21.15.181:0/2084992624 learned_addr learned my addr 172.21.15.181:0/2084992624 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:19.379 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0837fff640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 0x7f083815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f08377fe640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 0x7f0838152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 -- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 msgr2=0x7f0838152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 0x7f0838152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 -- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 msgr2=0x7f083815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 0x7f083815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 -- 172.21.15.181:0/2084992624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083806e630 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0837fff640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 0x7f083815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f08377fe640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 0x7f0838152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f0836ffd640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f082c011a70 tx=0x7f082c011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00fb50 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c00fcf0 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0838160440 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00fea0 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.396+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083806e8d0 con 0x7f083806f0d0 2024-03-29T04:10:19.380 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.397+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0838075900 con 0x7f083806f0d0 2024-03-29T04:10:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.398+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f082c012040 con 0x7f083806f0d0 2024-03-29T04:10:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.398+0000 7f081ffff640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 0x7f082406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.398+0000 7f08377fe640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 0x7f082406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.398+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f082c08bb10 con 0x7f083806f0d0 2024-03-29T04:10:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.399+0000 7f08377fe640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 0x7f082406ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0828033000 tx=0x7f0828009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.382 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.403+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f082c019030 con 0x7f083806f0d0 2024-03-29T04:10:19.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.643+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f0838162d10 con 0x7f083806f0d0 2024-03-29T04:10:19.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.645+0000 7f081ffff640 1 -- 172.21.15.181:0/2084992624 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f082c059d30 con 0x7f083806f0d0 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 msgr2=0x7f082406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 0x7f082406ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0828033000 tx=0x7f0828009030 comp rx=0 tx=0).stop 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 msgr2=0x7f083815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f082c011a70 tx=0x7f082c011f30 comp rx=0 tx=0).stop 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 shutdown_connections 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f083815aae0 0x7f083815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f083806f0d0 0x7f083815a280 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f082406c560 0x7f082406ea20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 --2- 172.21.15.181:0/2084992624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08200039b0 0x7f0838152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 >> 172.21.15.181:0/2084992624 conn(0x7f0838071d00 msgr2=0x7f08380cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 shutdown_connections 2024-03-29T04:10:19.628 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.647+0000 7f083daf1640 1 -- 172.21.15.181:0/2084992624 wait complete. 2024-03-29T04:10:19.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:19.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.758+0000 7fbdd8070640 1 Processor -- start 2024-03-29T04:10:19.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.758+0000 7fbdd8070640 1 -- start start 2024-03-29T04:10:19.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd8070640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbdd006db10 0x7fbdd006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd8070640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00ce840 con 0x7fbdd0072230 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd8070640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00ceb20 con 0x7fbdd0072610 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd8070640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd0158f10 con 0x7fbdd006db10 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd5e0c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbdd0072230 0x7fbdd006bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54508/0 (socket says 172.21.15.181:54508) 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.759+0000 7fbdd5e0c640 1 -- 172.21.15.181:0/508084919 learned_addr learned my addr 172.21.15.181:0/508084919 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3218562991 0 0) 0x7fbdd00ce840 con 0x7fbdd0072230 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd660d640 1 --2- 172.21.15.181:0/508084919 >> v2:172.21.15.187:3300/0 conn(0x7fbdd006db10 0x7fbdd006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdbc003520 con 0x7fbdd0072230 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3078927820 0 0) 0x7fbdbc003520 con 0x7fbdd0072230 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 >> v2:172.21.15.187:3300/0 conn(0x7fbdd006db10 msgr2=0x7fbdd006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 --2- 172.21.15.181:0/508084919 >> v2:172.21.15.187:3300/0 conn(0x7fbdd006db10 0x7fbdd006df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 >> v1:172.21.15.187:6789/0 conn(0x7fbdd0072610 legacy=0x7fbdd015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd015ec00 con 0x7fbdd0072230 2024-03-29T04:10:19.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fbdcc0019f0 con 0x7fbdd0072230 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd660d640 1 --2- 172.21.15.181:0/508084919 >> v2:172.21.15.187:3300/0 conn(0x7fbdd006db10 0x7fbdd006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 >> v1:172.21.15.181:6789/0 conn(0x7fbdd0072230 legacy=0x7fbdd006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 0x7fbdbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdbc004800 0x7fbdbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0051c0 0x7fbdbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0158f10 con 0x7fbdbc0039b0 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00c110 con 0x7fbdbc0051c0 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbdbc00c310 con 0x7fbdbc004800 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 0x7fbdbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd660d640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0051c0 0x7fbdbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd560b640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdbc004800 0x7fbdbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 -- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdbc004800 msgr2=0x7fbdbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdbc004800 0x7fbdbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 -- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0051c0 msgr2=0x7fbdbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0051c0 0x7fbdbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd660d640 1 -- 172.21.15.181:0/508084919 reap_dead start 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd5e0c640 1 -- 172.21.15.181:0/508084919 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc003520 con 0x7fbdbc0039b0 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.760+0000 7fbdd560b640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdbc004800 0x7fbdbc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.761+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 0x7fbdbc003dd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00a840 tx=0x7fbdbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4eb97749a9dd011 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.761+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00a460 con 0x7fbdbc0039b0 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.761+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc034650 con 0x7fbdbc0039b0 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.761+0000 7fbdd4e0a640 1 -- 172.21.15.181:0/508084919 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc034a80 con 0x7fbdbc0039b0 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 -- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 msgr2=0x7fbdbc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 0x7fbdbc003dd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00a840 tx=0x7fbdbc005830 comp rx=0 tx=0).stop 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 -- 172.21.15.181:0/508084919 shutdown_connections 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 --2- 172.21.15.181:0/508084919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdbc0039b0 0x7fbdbc003dd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 -- 172.21.15.181:0/508084919 >> 172.21.15.181:0/508084919 conn(0x7fbdd00d0cb0 msgr2=0x7fbdd00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 -- 172.21.15.181:0/508084919 shutdown_connections 2024-03-29T04:10:19.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.762+0000 7fbdd8070640 1 -- 172.21.15.181:0/508084919 wait complete. 2024-03-29T04:10:19.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.763+0000 7fbdd8070640 1 Processor -- start 2024-03-29T04:10:19.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.763+0000 7fbdd8070640 1 -- start start 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 0x7fbdd0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 0x7fbdd00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd006fa90 con 0x7fbdd00aee30 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd015ef70 con 0x7fbdbc0039b0 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd00ccff0 con 0x7fbdd014d3f0 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54902/0 (socket says 172.21.15.181:54902) 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 -- 172.21.15.181:0/1480498766 learned_addr learned my addr 172.21.15.181:0/1480498766 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 0x7fbdd0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd560b640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 0x7fbdd00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 -- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 msgr2=0x7fbdd00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 0x7fbdd00ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 -- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 msgr2=0x7fbdd0160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 0x7fbdd0160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 -- 172.21.15.181:0/1480498766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd014d850 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 0x7fbdd0160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd560b640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 0x7fbdd00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd660d640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011490 tx=0x7fbdc8011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800f040 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc8004020 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8004410 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd015ff80 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.764+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd0160540 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.765+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd00ce460 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.766+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbdc801e020 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.766+0000 7fbdc67fc640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 0x7fbdb406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.766+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fbdc808adf0 con 0x7fbdd00aee30 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.766+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 0x7fbdb406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.767+0000 7fbdd5e0c640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 0x7fbdb406e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034000 tx=0x7fbdcc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:19.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.771+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbdd00ce460 con 0x7fbdd00aee30 2024-03-29T04:10:19.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.947+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdd00d1200 con 0x7fbdd00aee30 2024-03-29T04:10:19.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.947+0000 7fbdc67fc640 1 -- 172.21.15.181:0/1480498766 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13176 (secure 0 0 0) 0x7fbdd00d1200 con 0x7fbdd00aee30 2024-03-29T04:10:19.927 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:19.927 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":35,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:17.049421+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.949+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 msgr2=0x7fbdb406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.949+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 0x7fbdb406e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc034000 tx=0x7fbdcc009030 comp rx=0 tx=0).stop 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.949+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 msgr2=0x7fbdd00b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.949+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011490 tx=0x7fbdc8011950 comp rx=0 tx=0).stop 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 shutdown_connections 2024-03-29T04:10:19.929 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdd00aee30 0x7fbdd00b41d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fbdb406c530 0x7fbdb406e9f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbdd014d3f0 0x7fbdd00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 --2- 172.21.15.181:0/1480498766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbdbc0039b0 0x7fbdd0160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 >> 172.21.15.181:0/1480498766 conn(0x7fbdd00d0cb0 msgr2=0x7fbdd00b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 shutdown_connections 2024-03-29T04:10:19.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:19.950+0000 7fbdd8070640 1 -- 172.21.15.181:0/1480498766 wait complete. 2024-03-29T04:10:19.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:10:19.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:10:20.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.065+0000 7efc12782640 1 Processor -- start 2024-03-29T04:10:20.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.065+0000 7efc12782640 1 -- start start 2024-03-29T04:10:20.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc12782640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c1605a0 0x7efc0c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc12782640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c0ce9c0 con 0x7efc0c06d8f0 2024-03-29T04:10:20.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc12782640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c160b50 con 0x7efc0c13f0c0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc12782640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c160e30 con 0x7efc0c1605a0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0b7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efc0c06d8f0 0x7efc0c15fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54514/0 (socket says 172.21.15.181:54514) 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0b7fe640 1 -- 172.21.15.181:0/1717210088 learned_addr learned my addr 172.21.15.181:0/1717210088 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc10d1f640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c1605a0 0x7efc0c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 885680080 0 0) 0x7efbfc001110 con 0x7efc0c06d8f0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbf4003520 con 0x7efc0c06d8f0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413437265 0 0) 0x7efc0c160b50 con 0x7efc0c13f0c0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbfc001110 con 0x7efc0c13f0c0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726907781 0 0) 0x7efbf4003520 con 0x7efc0c06d8f0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c1605a0 msgr2=0x7efc0c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c1605a0 0x7efc0c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 >> v1:172.21.15.187:6789/0 conn(0x7efc0c13f0c0 legacy=0x7efc0c06d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c13fa40 con 0x7efc0c06d8f0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7efbfc001860 con 0x7efc0c06d8f0 2024-03-29T04:10:20.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 >> v1:172.21.15.181:6789/0 conn(0x7efc0c06d8f0 legacy=0x7efc0c15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efbf4003b70 0x7efbf4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf4004800 0x7efbf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4005210 0x7efbf400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c160e30 con 0x7efbf4004800 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400c140 con 0x7efbf4005210 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.066+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efbf400c340 con 0x7efbf4003b70 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc10d1f640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c1605a0 0x7efc0c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf4004800 0x7efbf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0bfff640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efbf4003b70 0x7efbf4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc10d1f640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4005210 0x7efbf400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 -- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efbf4003b70 msgr2=0x7efbf4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efbf4003b70 0x7efbf4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 -- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4005210 msgr2=0x7efbf400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:20.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4005210 0x7efbf400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 -- 172.21.15.181:0/1717210088 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf4003520 con 0x7efbf4004800 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc10d1f640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4005210 0x7efbf400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.067+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf4004800 0x7efbf4004c80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc0c142980 tx=0x7efbf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54f15f3011b0c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc007e00 con 0x7efbf4004800 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc030070 con 0x7efbf4004800 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc0affd640 1 -- 172.21.15.181:0/1717210088 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc02cd60 con 0x7efbf4004800 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 -- 172.21.15.181:0/1717210088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf4004800 msgr2=0x7efbf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 --2- 172.21.15.181:0/1717210088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efbf4004800 0x7efbf4004c80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc0c142980 tx=0x7efbf4005830 comp rx=0 tx=0).stop 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc10d1f640 1 -- 172.21.15.181:0/1717210088 reap_dead start 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 -- 172.21.15.181:0/1717210088 shutdown_connections 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 -- 172.21.15.181:0/1717210088 >> 172.21.15.181:0/1717210088 conn(0x7efc0c0b5b00 msgr2=0x7efc0c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 -- 172.21.15.181:0/1717210088 shutdown_connections 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.068+0000 7efc12782640 1 -- 172.21.15.181:0/1717210088 wait complete. 2024-03-29T04:10:20.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.069+0000 7efc12782640 1 Processor -- start 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 -- start start 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 0x7efc0c13c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c154c70 0x7efc0c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c13f4a0 con 0x7efc0c141e80 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c140000 con 0x7efbf4004800 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc12782640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c13f620 con 0x7efc0c154c70 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc0b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc0b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54920/0 (socket says 172.21.15.181:54920) 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.070+0000 7efc0b7fe640 1 -- 172.21.15.181:0/2713878479 learned_addr learned my addr 172.21.15.181:0/2713878479 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0bfff640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 0x7efc0c13c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc10d1f640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c154c70 0x7efc0c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 -- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c154c70 msgr2=0x7efc0c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c154c70 0x7efc0c1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 -- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 msgr2=0x7efc0c13c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 0x7efc0c13c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 -- 172.21.15.181:0/2713878479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1435f0 con 0x7efc0c141e80 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0bfff640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 0x7efc0c13c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc0b7fe640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00c3e0 tx=0x7efc0c13fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc045070 con 0x7efc0c141e80 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc004bf0 con 0x7efc0c141e80 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc007e00 con 0x7efc0c141e80 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c155630 con 0x7efc0c141e80 2024-03-29T04:10:20.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.071+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1428b0 con 0x7efc0c141e80 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.072+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c075890 con 0x7efc0c141e80 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.072+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efbfc007070 con 0x7efc0c141e80 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.073+0000 7efc08ff9640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 0x7efbf806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.073+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7efbfc0ac480 con 0x7efc0c141e80 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.073+0000 7efc0bfff640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 0x7efbf806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.073+0000 7efc0bfff640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 0x7efbf806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc00007fd0 tx=0x7efc00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.077+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc0c0ce9c0 con 0x7efc0c141e80 2024-03-29T04:10:20.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.251+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc0c06c2c0 con 0x7efc0c141e80 2024-03-29T04:10:20.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.252+0000 7efc08ff9640 1 -- 172.21.15.181:0/2713878479 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7efbfc07a6f0 con 0x7efc0c141e80 2024-03-29T04:10:20.231 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:10:20.233 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 msgr2=0x7efbf806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 0x7efbf806e9f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc00007fd0 tx=0x7efc00009040 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 msgr2=0x7efc0c156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00c3e0 tx=0x7efc0c13fc10 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 shutdown_connections 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efbf806c530 0x7efbf806e9f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc0c154c70 0x7efc0c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc0c141e80 0x7efc0c156a00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 --2- 172.21.15.181:0/2713878479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efbf4004800 0x7efc0c13c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 >> 172.21.15.181:0/2713878479 conn(0x7efc0c0b5b00 msgr2=0x7efc0c15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 shutdown_connections 2024-03-29T04:10:20.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.254+0000 7efc12782640 1 -- 172.21.15.181:0/2713878479 wait complete. 2024-03-29T04:10:20.244 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T04:10:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.366+0000 7f5307732640 1 Processor -- start 2024-03-29T04:10:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.366+0000 7f5307732640 1 -- start start 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5307732640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 0x7f53000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5307732640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f530015f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5307732640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5300144e10 con 0x7f5300072340 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5307732640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f53000cd610 con 0x7f53001445e0 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5307732640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53001449c0 con 0x7f53000cbda0 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5305ccf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f530015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f53054ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 0x7f53000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5304ccd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f5300072340 0x7f53000cb690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:58158/0 (socket says 172.21.15.181:58158) 2024-03-29T04:10:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f5304ccd640 1 -- 172.21.15.181:0/2503955339 learned_addr learned my addr 172.21.15.181:0/2503955339 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.367+0000 7f52f7fff640 1 -- 172.21.15.181:0/2503955339 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207563135 0 0) 0x7f5300144e10 con 0x7f5300072340 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f52f7fff640 1 -- 172.21.15.181:0/2503955339 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52ec003520 con 0x7f5300072340 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f5305ccf640 1 -- 172.21.15.181:0/2503955339 >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 msgr2=0x7f53000cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f5305ccf640 1 --2- 172.21.15.181:0/2503955339 >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 0x7f53000cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f5305ccf640 1 -- 172.21.15.181:0/2503955339 >> v1:172.21.15.187:6789/0 conn(0x7f5300072340 legacy=0x7f53000cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f5305ccf640 1 -- 172.21.15.181:0/2503955339 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000caa80 con 0x7f53000cbda0 2024-03-29T04:10:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f53054ce640 1 --2- 172.21.15.181:0/2503955339 >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 0x7f53000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.368+0000 7f5305ccf640 1 --2- 172.21.15.181:0/2503955339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f530015f240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c7f26acadc4afd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f52f7fff640 1 -- 172.21.15.181:0/2503955339 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8039070 con 0x7f53000cbda0 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f52f7fff640 1 -- 172.21.15.181:0/2503955339 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f802cc50 con 0x7f53000cbda0 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f52f7fff640 1 -- 172.21.15.181:0/2503955339 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8034440 con 0x7f53000cbda0 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 -- 172.21.15.181:0/2503955339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 msgr2=0x7f530015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 --2- 172.21.15.181:0/2503955339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f530015f240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802bea0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 -- 172.21.15.181:0/2503955339 shutdown_connections 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 --2- 172.21.15.181:0/2503955339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f530015f240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 --2- 172.21.15.181:0/2503955339 >> v2:172.21.15.187:3300/0 conn(0x7f53001445e0 0x7f53000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 -- 172.21.15.181:0/2503955339 >> 172.21.15.181:0/2503955339 conn(0x7f53000b1c90 msgr2=0x7f53000b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 -- 172.21.15.181:0/2503955339 shutdown_connections 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.369+0000 7f5307732640 1 -- 172.21.15.181:0/2503955339 wait complete. 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.370+0000 7f5307732640 1 Processor -- start 2024-03-29T04:10:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.370+0000 7f5307732640 1 -- start start 2024-03-29T04:10:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f53000b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53001445e0 0x7f530013b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f530015edb0 con 0x7f5300072340 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53000cf5b0 con 0x7f53001445e0 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5307732640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53000cb040 con 0x7f53000cbda0 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54930/0 (socket says 172.21.15.181:54930) 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 -- 172.21.15.181:0/2953670723 learned_addr learned my addr 172.21.15.181:0/2953670723 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5304ccd640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f53000b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5305ccf640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53001445e0 0x7f530013b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 -- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 msgr2=0x7f53000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f53000b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 -- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53001445e0 msgr2=0x7f530013b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53001445e0 0x7f530013b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f53054ce640 1 -- 172.21.15.181:0/2953670723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000b51b0 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.371+0000 7f5304ccd640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f53000b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f53054ce640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00e990 tx=0x7f52fc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc017070 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc00c9e0 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53000b5450 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00cdb0 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.372+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53000b3a70 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.373+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5300075a10 con 0x7f5300072340 2024-03-29T04:10:20.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.373+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52fc01a070 con 0x7f5300072340 2024-03-29T04:10:20.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.374+0000 7f52f5ffb640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 0x7f52e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.374+0000 7f5304ccd640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 0x7f52e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.374+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f52fc08b320 con 0x7f5300072340 2024-03-29T04:10:20.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.374+0000 7f5304ccd640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 0x7f52e406e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52f0007fd0 tx=0x7f52f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.378+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f52fc0594b0 con 0x7f5300072340 2024-03-29T04:10:20.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.553+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f530013f190 con 0x7f5300072340 2024-03-29T04:10:20.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.554+0000 7f52f5ffb640 1 -- 172.21.15.181:0/2953670723 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f52fc05d7b0 con 0x7f5300072340 2024-03-29T04:10:20.533 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:20.533 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2024-03-29T04:10:20.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-03-29T04:10:20.534 INFO:teuthology.orchestra.run.smithi181.stdout: "poolnum": 1, 2024-03-29T04:10:20.534 INFO:teuthology.orchestra.run.smithi181.stdout: "poolname": ".mgr" 2024-03-29T04:10:20.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:20.534 INFO:teuthology.orchestra.run.smithi181.stdout:] 2024-03-29T04:10:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 msgr2=0x7f52e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 0x7f52e406e9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52f0007fd0 tx=0x7f52f0009040 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 msgr2=0x7f5300134dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00e990 tx=0x7f52fc00ee50 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 shutdown_connections 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53001445e0 0x7f530013b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f52e406c530 0x7f52e406e9f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53000cbda0 0x7f53000b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 --2- 172.21.15.181:0/2953670723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5300072340 0x7f5300134dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 >> 172.21.15.181:0/2953670723 conn(0x7f53000b1c90 msgr2=0x7f5300072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 shutdown_connections 2024-03-29T04:10:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.556+0000 7f5307732640 1 -- 172.21.15.181:0/2953670723 wait complete. 2024-03-29T04:10:20.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T04:10:20.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T04:10:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.671+0000 7fe2f461b640 1 Processor -- start 2024-03-29T04:10:20.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.671+0000 7fe2f461b640 1 -- start start 2024-03-29T04:10:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f461b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2ec0ccd90 0x7fe2ec0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f461b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f461b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ec0cc3a0 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f461b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec0cdbf0 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f461b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec0cfee0 con 0x7fe2ec0d10b0 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f1bb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f23b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2ec0ccd90 0x7fe2ec0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.672+0000 7fe2f1bb6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:36750/0 (socket says 172.21.15.181:36750) 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f1bb6640 1 -- 172.21.15.181:0/953796036 learned_addr learned my addr 172.21.15.181:0/953796036 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f23b7640 1 -- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 msgr2=0x7fe2ec0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f23b7640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f23b7640 1 -- 172.21.15.181:0/953796036 >> v1:172.21.15.187:6789/0 conn(0x7fe2ec0cfa20 legacy=0x7fe2ec0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f23b7640 1 -- 172.21.15.181:0/953796036 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec0ce0d0 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.673+0000 7fe2f1bb6640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f23b7640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0ccd90 0x7fe2ec0cd170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097a0 tx=0x7fe2e802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc07c1427226467f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f13b5640 1 -- 172.21.15.181:0/953796036 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8035020 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f13b5640 1 -- 172.21.15.181:0/953796036 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e8031830 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f13b5640 1 -- 172.21.15.181:0/953796036 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8031c00 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 -- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0ccd90 msgr2=0x7fe2ec0cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0ccd90 0x7fe2ec0cd170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097a0 tx=0x7fe2e802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 -- 172.21.15.181:0/953796036 shutdown_connections 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0d10b0 0x7fe2ec0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 --2- 172.21.15.181:0/953796036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0ccd90 0x7fe2ec0cd170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 -- 172.21.15.181:0/953796036 >> 172.21.15.181:0/953796036 conn(0x7fe2ec0d0380 msgr2=0x7fe2ec0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:20.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 -- 172.21.15.181:0/953796036 shutdown_connections 2024-03-29T04:10:20.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.674+0000 7fe2f461b640 1 -- 172.21.15.181:0/953796036 wait complete. 2024-03-29T04:10:20.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 Processor -- start 2024-03-29T04:10:20.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 -- start start 2024-03-29T04:10:20.658 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0ccd90 0x7fe2ec1e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 0x7fe2ec15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0ca280 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0ca100 con 0x7fe2ec0d10b0 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f461b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec0cc880 con 0x7fe2ec0ccd90 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f1bb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f1bb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54946/0 (socket says 172.21.15.181:54946) 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.676+0000 7fe2f1bb6640 1 -- 172.21.15.181:0/3771079392 learned_addr learned my addr 172.21.15.181:0/3771079392 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f23b7640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0ccd90 0x7fe2ec1e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f2bb8640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 0x7fe2ec15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 -- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0ccd90 msgr2=0x7fe2ec1e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0ccd90 0x7fe2ec1e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.659 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 -- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 msgr2=0x7fe2ec15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 0x7fe2ec15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 -- 172.21.15.181:0/3771079392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec1606c0 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f2bb8640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 0x7fe2ec15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f1bb6640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e9e0 tx=0x7fe2dc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00c9d0 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00cd80 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec160960 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.677+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec160f20 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.678+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ec075900 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.678+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe2dc01a070 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.679+0000 7fe2e2ffd640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 0x7fe2d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.679+0000 7fe2f23b7640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 0x7fe2d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.679+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fe2dc08b310 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.660 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.679+0000 7fe2f23b7640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 0x7fe2d006e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8036000 tx=0x7fe2e8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:20.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.683+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe2dc059b80 con 0x7fe2ec0cfa20 2024-03-29T04:10:20.838 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:20.858+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe2ec0c8150 con 0x7fe2ec0cfa20 2024-03-29T04:10:21.259 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.279+0000 7fe2e2ffd640 1 -- 172.21.15.181:0/3771079392 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fe2dc089e20 con 0x7fe2ec0cfa20 2024-03-29T04:10:21.259 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' created 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 msgr2=0x7fe2d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 0x7fe2d006e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8036000 tx=0x7fe2e8008030 comp rx=0 tx=0).stop 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 msgr2=0x7fe2ec15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e9e0 tx=0x7fe2dc00eea0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 shutdown_connections 2024-03-29T04:10:21.261 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2ec0d10b0 0x7fe2ec15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ec0cfa20 0x7fe2ec15d510 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe2d006c530 0x7fe2d006e9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 --2- 172.21.15.181:0/3771079392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2ec0ccd90 0x7fe2ec1e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 >> 172.21.15.181:0/3771079392 conn(0x7fe2ec0d0380 msgr2=0x7fe2ec0cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 shutdown_connections 2024-03-29T04:10:21.262 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.282+0000 7fe2f461b640 1 -- 172.21.15.181:0/3771079392 wait complete. 2024-03-29T04:10:21.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T04:10:21.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.404+0000 7f4c064f2640 1 Processor -- start 2024-03-29T04:10:21.383 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.404+0000 7f4c064f2640 1 -- start start 2024-03-29T04:10:21.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c064f2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4c00144c40 0x7f4c000cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c064f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c000c58c0 0x7f4c000cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c064f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c000c5cf0 con 0x7f4c000ce150 2024-03-29T04:10:21.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c064f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f4c00145020 con 0x7f4c000c58c0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c064f2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4c000c5f80 con 0x7f4c00144c40 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c000c58c0 0x7f4c000cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4bff7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4c000ce150 0x7f4c000c51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54528/0 (socket says 172.21.15.181:54528) 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4bff7fe640 1 -- 172.21.15.181:0/355650823 learned_addr learned my addr 172.21.15.181:0/355650823 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c000c58c0 0x7f4c000cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54960/0 (socket says 172.21.15.181:54960) 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4bfffff640 1 --2- 172.21.15.181:0/355650823 >> v2:172.21.15.187:3300/0 conn(0x7f4c00144c40 0x7f4c000cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 -- 172.21.15.181:0/355650823 >> v2:172.21.15.187:3300/0 conn(0x7f4c00144c40 msgr2=0x7f4c000cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 --2- 172.21.15.181:0/355650823 >> v2:172.21.15.187:3300/0 conn(0x7f4c00144c40 0x7f4c000cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 -- 172.21.15.181:0/355650823 >> v1:172.21.15.181:6789/0 conn(0x7f4c000ce150 legacy=0x7f4c000c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4bfeffd640 1 -- 172.21.15.181:0/355650823 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608922340 0 0) 0x7f4c000c5cf0 con 0x7f4c000ce150 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.405+0000 7f4c04a8f640 1 -- 172.21.15.181:0/355650823 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c000c6260 con 0x7f4c000c58c0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.406+0000 7f4c04a8f640 1 --2- 172.21.15.181:0/355650823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c000c58c0 0x7f4c000cc180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4c000cffa0 tx=0x7f4bf802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1444e2931c055df server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.406+0000 7f4bfeffd640 1 -- 172.21.15.181:0/355650823 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8039070 con 0x7f4c000c58c0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.406+0000 7f4bfeffd640 1 -- 172.21.15.181:0/355650823 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf802cc80 con 0x7f4c000c58c0 2024-03-29T04:10:21.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.406+0000 7f4bfeffd640 1 -- 172.21.15.181:0/355650823 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8034660 con 0x7f4c000c58c0 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 -- 172.21.15.181:0/355650823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c000c58c0 msgr2=0x7f4c000cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 --2- 172.21.15.181:0/355650823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c000c58c0 0x7f4c000cc180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4c000cffa0 tx=0x7f4bf802bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 -- 172.21.15.181:0/355650823 shutdown_connections 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 --2- 172.21.15.181:0/355650823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c000c58c0 0x7f4c000cc180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 --2- 172.21.15.181:0/355650823 >> v2:172.21.15.187:3300/0 conn(0x7f4c00144c40 0x7f4c000cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 -- 172.21.15.181:0/355650823 >> 172.21.15.181:0/355650823 conn(0x7f4c000b1c90 msgr2=0x7f4c000b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 -- 172.21.15.181:0/355650823 shutdown_connections 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.407+0000 7f4c064f2640 1 -- 172.21.15.181:0/355650823 wait complete. 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.408+0000 7f4c064f2640 1 Processor -- start 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.408+0000 7f4c064f2640 1 -- start start 2024-03-29T04:10:21.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c000c58c0 0x7f4c0015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 0x7f4c000b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4c000cb880 con 0x7f4c00144c40 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c000cbc60 con 0x7f4c000c58c0 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c000cba00 con 0x7f4c000ce150 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54974/0 (socket says 172.21.15.181:54974) 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 -- 172.21.15.181:0/1694876114 learned_addr learned my addr 172.21.15.181:0/1694876114 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bfffff640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c000c58c0 0x7f4c0015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bff7fe640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 0x7f4c000b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 -- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 msgr2=0x7f4c000b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 0x7f4c000b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 -- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c000c58c0 msgr2=0x7f4c0015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c000c58c0 0x7f4c0015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:21.392 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 -- 172.21.15.181:0/1694876114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c000b4800 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bff7fe640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 0x7f4c000b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c04a8f640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8037040 tx=0x7f4bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8039050 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf8045070 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8009450 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c000b4ae0 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.409+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c000b50a0 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.410+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c00075890 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.411+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4bf803e850 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.411+0000 7f4bfcff9640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 0x7f4bec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.411+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f4bf80ae270 con 0x7f4c00144c40 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.411+0000 7f4bfffff640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 0x7f4bec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:21.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.412+0000 7f4bfffff640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 0x7f4bec06e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009a30 tx=0x7f4bf0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:21.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.416+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4bf8032e20 con 0x7f4c00144c40 2024-03-29T04:10:21.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:21.631+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4c000b5de0 con 0x7f4c00144c40 2024-03-29T04:10:22.266 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.286+0000 7f4bfcff9640 1 -- 172.21.15.181:0/1694876114 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f4bf807ca00 con 0x7f4c00144c40 2024-03-29T04:10:22.266 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' created 2024-03-29T04:10:22.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.289+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 msgr2=0x7f4bec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.289+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 0x7f4bec06e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009a30 tx=0x7f4bf0008040 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 msgr2=0x7f4c000b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8037040 tx=0x7f4bf8004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 shutdown_connections 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c00144c40 0x7f4c000b3dd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4bec06c530 0x7f4bec06e9f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c000ce150 0x7f4c000b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 --2- 172.21.15.181:0/1694876114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c000c58c0 0x7f4c0015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 >> 172.21.15.181:0/1694876114 conn(0x7f4c000b1c90 msgr2=0x7f4c000cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 shutdown_connections 2024-03-29T04:10:22.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.290+0000 7f4c064f2640 1 -- 172.21.15.181:0/1694876114 wait complete. 2024-03-29T04:10:22.291 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T04:10:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 Processor -- start 2024-03-29T04:10:22.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 -- start start 2024-03-29T04:10:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfb00ba090 0x7fcfb00d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb00b66e0 con 0x7fcfb006c620 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb00d2ff0 con 0x7fcfb00ba090 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.420+0000 7fcfb503c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb00d32d0 con 0x7fcfb00b5dc0 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfb00ba090 0x7fcfb00d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcfb00ba090 0x7fcfb00d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54978/0 (socket says 172.21.15.181:54978) 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 -- 172.21.15.181:0/3889094780 learned_addr learned my addr 172.21.15.181:0/3889094780 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfaed9d640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 -- 172.21.15.181:0/3889094780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 msgr2=0x7fcfb00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb00b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 -- 172.21.15.181:0/3889094780 >> v1:172.21.15.187:6789/0 conn(0x7fcfb006c620 legacy=0x7fcfb006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfa7fff640 1 -- 172.21.15.181:0/3889094780 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb00d35b0 con 0x7fcfb00ba090 2024-03-29T04:10:22.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.421+0000 7fcfaed9d640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:22.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfa7fff640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb00d2680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009870 tx=0x7fcf9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f24d5fb1a265145 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfae59c640 1 -- 172.21.15.181:0/3889094780 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98035020 con 0x7fcfb00ba090 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfae59c640 1 -- 172.21.15.181:0/3889094780 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf98031640 con 0x7fcfb00ba090 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfae59c640 1 -- 172.21.15.181:0/3889094780 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9802d040 con 0x7fcfb00ba090 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 -- 172.21.15.181:0/3889094780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 msgr2=0x7fcfb00d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb00d2680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009870 tx=0x7fcf9802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 -- 172.21.15.181:0/3889094780 shutdown_connections 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb00d2680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 --2- 172.21.15.181:0/3889094780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb00b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 -- 172.21.15.181:0/3889094780 >> 172.21.15.181:0/3889094780 conn(0x7fcfb00678f0 msgr2=0x7fcfb00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 -- 172.21.15.181:0/3889094780 shutdown_connections 2024-03-29T04:10:22.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.422+0000 7fcfb503c640 1 -- 172.21.15.181:0/3889094780 wait complete. 2024-03-29T04:10:22.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 Processor -- start 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 -- start start 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb006c620 0x7fcfb01518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb0160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb00d4090 con 0x7fcfb00ba090 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb00d3b70 con 0x7fcfb006c620 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.424+0000 7fcfb503c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb00b83e0 con 0x7fcfb00b5dc0 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54990/0 (socket says 172.21.15.181:54990) 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 -- 172.21.15.181:0/4197991313 learned_addr learned my addr 172.21.15.181:0/4197991313 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaed9d640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb006c620 0x7fcfb01518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfa7fff640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb0160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 -- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 msgr2=0x7fcfb0160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb0160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 -- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb006c620 msgr2=0x7fcfb01518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb006c620 0x7fcfb01518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 -- 172.21.15.181:0/4197991313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb0160990 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfa7fff640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb0160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfaf59e640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c780 tx=0x7fcfa800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8004b60 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa8004d00 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800aa90 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb0160c70 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.425+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb0161230 con 0x7fcfb00ba090 2024-03-29T04:10:22.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.426+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb0075890 con 0x7fcfb00ba090 2024-03-29T04:10:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.427+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcfa800ac30 con 0x7fcfb00ba090 2024-03-29T04:10:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.427+0000 7fcfa77fe640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.427+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7fcfa808ba60 con 0x7fcfb00ba090 2024-03-29T04:10:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.427+0000 7fcfaed9d640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:22.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.428+0000 7fcfaed9d640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 0x7fcf9006e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000ab60 tx=0x7fcfa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:22.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.432+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcfa8020070 con 0x7fcfb00ba090 2024-03-29T04:10:22.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:22.644+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcfb00b8860 con 0x7fcfb00ba090 2024-03-29T04:10:23.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.287+0000 7fcfa77fe640 1 -- 172.21.15.181:0/4197991313 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fcfa8059f50 con 0x7fcfb00ba090 2024-03-29T04:10:23.267 INFO:teuthology.orchestra.run.smithi181.stderr:new fs with metadata pool 5 and data pool 6 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 msgr2=0x7fcf9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 0x7fcf9006e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000ab60 tx=0x7fcfa0008040 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 msgr2=0x7fcfb015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c780 tx=0x7fcfa800cc40 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 shutdown_connections 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfb00ba090 0x7fcfb015c190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcfb00b5dc0 0x7fcfb0160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 --2- 172.21.15.181:0/4197991313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfb006c620 0x7fcfb01518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.292+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 >> 172.21.15.181:0/4197991313 conn(0x7fcfb00678f0 msgr2=0x7fcfb015db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.293+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 shutdown_connections 2024-03-29T04:10:23.274 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.295+0000 7fcfb503c640 1 -- 172.21.15.181:0/4197991313 wait complete. 2024-03-29T04:10:23.307 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:23.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.431+0000 7f6552bf4640 1 Processor -- start 2024-03-29T04:10:23.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.431+0000 7f6552bf4640 1 -- start start 2024-03-29T04:10:23.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c1445e0 0x7f654c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f654c072340 0x7f654c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f654c0727a0 0x7f654c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f654c0cd610 con 0x7f654c072340 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f654c1449c0 con 0x7f654c0727a0 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6552bf4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f654c0cf5b0 con 0x7f654c1445e0 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f654c072340 0x7f654c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f654c072340 0x7f654c0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:54996/0 (socket says 172.21.15.181:54996) 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 -- 172.21.15.181:0/3224689595 learned_addr learned my addr 172.21.15.181:0/3224689595 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f6550990640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c1445e0 0x7f654c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 -- 172.21.15.181:0/3224689595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c1445e0 msgr2=0x7f654c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c1445e0 0x7f654c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 -- 172.21.15.181:0/3224689595 >> v2:172.21.15.187:3300/0 conn(0x7f654c0727a0 msgr2=0x7f654c0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 --2- 172.21.15.181:0/3224689595 >> v2:172.21.15.187:3300/0 conn(0x7f654c0727a0 0x7f654c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.432+0000 7f654bfff640 1 -- 172.21.15.181:0/3224689595 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c144e10 con 0x7f654c072340 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.433+0000 7f654bfff640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c072340 0x7f654c0cb690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f654c074110 tx=0x7f653802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c4375a5f0f1bd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.433+0000 7f654b7fe640 1 -- 172.21.15.181:0/3224689595 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538035020 con 0x7f654c072340 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.433+0000 7f654b7fe640 1 -- 172.21.15.181:0/3224689595 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6538031640 con 0x7f654c072340 2024-03-29T04:10:23.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.433+0000 7f654b7fe640 1 -- 172.21.15.181:0/3224689595 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653802d040 con 0x7f654c072340 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.433+0000 7f6552bf4640 1 -- 172.21.15.181:0/3224689595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c072340 msgr2=0x7f654c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c072340 0x7f654c0cb690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f654c074110 tx=0x7f653802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 -- 172.21.15.181:0/3224689595 shutdown_connections 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 --2- 172.21.15.181:0/3224689595 >> v2:172.21.15.187:3300/0 conn(0x7f654c0727a0 0x7f654c0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c072340 0x7f654c0cb690 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 --2- 172.21.15.181:0/3224689595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c1445e0 0x7f654c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 -- 172.21.15.181:0/3224689595 >> 172.21.15.181:0/3224689595 conn(0x7f654c0b1c90 msgr2=0x7f654c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 -- 172.21.15.181:0/3224689595 shutdown_connections 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.434+0000 7f6552bf4640 1 -- 172.21.15.181:0/3224689595 wait complete. 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.435+0000 7f6552bf4640 1 Processor -- start 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.435+0000 7f6552bf4640 1 -- start start 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c072340 0x7f654c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f654c0727a0 0x7f654c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f654c0cad60 con 0x7f654c1445e0 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f654c15edb0 con 0x7f654c0727a0 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f654c0cb040 con 0x7f654c072340 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55000/0 (socket says 172.21.15.181:55000) 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 -- 172.21.15.181:0/2308257975 learned_addr learned my addr 172.21.15.181:0/2308257975 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6550990640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c072340 0x7f654c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f654bfff640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f654c0727a0 0x7f654c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 -- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c072340 msgr2=0x7f654c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c072340 0x7f654c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 -- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f654c0727a0 msgr2=0x7f654c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f654c0727a0 0x7f654c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 -- 172.21.15.181:0/2308257975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c0c6c30 con 0x7f654c1445e0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6551191640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f654400b970 tx=0x7f654400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544019070 con 0x7f654c1445e0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654400cb60 con 0x7f654c1445e0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c0c6f10 con 0x7f654c1445e0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544014440 con 0x7f654c1445e0 2024-03-29T04:10:23.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.436+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c0c77b0 con 0x7f654c1445e0 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.437+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654c0c7300 con 0x7f654c1445e0 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.438+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f654401c070 con 0x7f654c1445e0 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.438+0000 7f65497fa640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 0x7f653006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.438+0000 7f6550990640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 0x7f653006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.438+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f654408b5d0 con 0x7f654c1445e0 2024-03-29T04:10:23.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.439+0000 7f6550990640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 0x7f653006e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.443+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6544059a60 con 0x7f654c1445e0 2024-03-29T04:10:23.601 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.621+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f654c075a10 con 0x7f654c1445e0 2024-03-29T04:10:23.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.622+0000 7f65497fa640 1 -- 172.21.15.181:0/2308257975 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16967 (secure 0 0 0) 0x7f6544059360 con 0x7f654c1445e0 2024-03-29T04:10:23.602 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:23.602 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:22.645488+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.624+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 msgr2=0x7f653006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.624+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 0x7f653006e9f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540008040 comp rx=0 tx=0).stop 2024-03-29T04:10:23.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.624+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 msgr2=0x7f654c0c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.624+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f654400b970 tx=0x7f654400be30 comp rx=0 tx=0).stop 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.624+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 shutdown_connections 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654c1445e0 0x7f654c0c6350 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f654c0727a0 0x7f654c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f653006c530 0x7f653006e9f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 --2- 172.21.15.181:0/2308257975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f654c072340 0x7f654c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 >> 172.21.15.181:0/2308257975 conn(0x7f654c0b1c90 msgr2=0x7f654c15f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 shutdown_connections 2024-03-29T04:10:23.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.625+0000 7f6552bf4640 1 -- 172.21.15.181:0/2308257975 wait complete. 2024-03-29T04:10:23.615 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.736+0000 7fd15403c640 1 Processor -- start 2024-03-29T04:10:23.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd15403c640 1 -- start start 2024-03-29T04:10:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd15403c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd14c0cd3b0 0x7fd14c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd15403c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c0724e0 con 0x7fd14c15a960 2024-03-29T04:10:23.717 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd15403c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c0727c0 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd15403c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fd14c0730f0 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.737+0000 7fd1515d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd14c0cd3b0 0x7fd14c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd151dd8640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd14c15a960 0x7fd14c071a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54530/0 (socket says 172.21.15.181:54530) 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd151dd8640 1 -- 172.21.15.181:0/3316822487 learned_addr learned my addr 172.21.15.181:0/3316822487 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd1515d7640 1 -- 172.21.15.181:0/3316822487 >> v1:172.21.15.187:6789/0 conn(0x7fd14c0cb3b0 legacy=0x7fd14c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd1515d7640 1 -- 172.21.15.181:0/3316822487 >> v1:172.21.15.181:6789/0 conn(0x7fd14c15a960 legacy=0x7fd14c071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd150dd6640 1 -- 172.21.15.181:0/3316822487 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2412651503 0 0) 0x7fd14c0724e0 con 0x7fd14c15a960 2024-03-29T04:10:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.738+0000 7fd1515d7640 1 -- 172.21.15.181:0/3316822487 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c072aa0 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd1515d7640 1 --2- 172.21.15.181:0/3316822487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c0cae70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00a940 tx=0x7fd13c02ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463f89b551d6bbf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd150dd6640 1 -- 172.21.15.181:0/3316822487 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c03a070 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd150dd6640 1 -- 172.21.15.181:0/3316822487 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c02dbe0 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd150dd6640 1 -- 172.21.15.181:0/3316822487 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00b040 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 -- 172.21.15.181:0/3316822487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cd3b0 msgr2=0x7fd14c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 --2- 172.21.15.181:0/3316822487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c0cae70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00a940 tx=0x7fd13c02ce30 comp rx=0 tx=0).stop 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 -- 172.21.15.181:0/3316822487 shutdown_connections 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 --2- 172.21.15.181:0/3316822487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c0cae70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 -- 172.21.15.181:0/3316822487 >> 172.21.15.181:0/3316822487 conn(0x7fd14c0b1c90 msgr2=0x7fd14c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 -- 172.21.15.181:0/3316822487 shutdown_connections 2024-03-29T04:10:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.739+0000 7fd15403c640 1 -- 172.21.15.181:0/3316822487 wait complete. 2024-03-29T04:10:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.740+0000 7fd15403c640 1 Processor -- start 2024-03-29T04:10:23.721 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.740+0000 7fd15403c640 1 -- start start 2024-03-29T04:10:23.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 0x7fd14c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c072180 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c15ad40 con 0x7fd14c0cd3b0 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd15403c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd14c0cca40 con 0x7fd14c15a960 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd151dd8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd151dd8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55020/0 (socket says 172.21.15.181:55020) 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd151dd8640 1 -- 172.21.15.181:0/2271776115 learned_addr learned my addr 172.21.15.181:0/2271776115 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd1525d9640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 0x7fd14c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.741+0000 7fd1515d7640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 -- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 msgr2=0x7fd14c0b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 0x7fd14c0b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 -- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd14c0cd3b0 msgr2=0x7fd14c14fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c14fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 -- 172.21.15.181:0/2271776115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c1f6480 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd1525d9640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 0x7fd14c0b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd151dd8640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c3f0 tx=0x7fd14800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1480074e0 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1480047e0 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148004c10 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c0b5010 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.742+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c1f5be0 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.743+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14c075a10 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.743+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd148011070 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.744+0000 7fd1427fc640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 0x7fd13006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.744+0000 7fd1515d7640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 0x7fd13006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.744+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd14808b3c0 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.744+0000 7fd1515d7640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 0x7fd13006e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd13c037000 tx=0x7fd13c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:23.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.748+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd148059850 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.922+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd14c1f2060 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.923+0000 7fd1427fc640 1 -- 172.21.15.181:0/2271776115 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16967 (secure 0 0 0) 0x7fd148059150 con 0x7fd14c0cb3b0 2024-03-29T04:10:23.903 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:23.903 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":38,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:22.645488+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 msgr2=0x7fd13006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 0x7fd13006e9f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd13c037000 tx=0x7fd13c008040 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 msgr2=0x7fd14c150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c3f0 tx=0x7fd14800c8b0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 shutdown_connections 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd13006c530 0x7fd13006e9f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd14c15a960 0x7fd14c0b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd14c0cd3b0 0x7fd14c14fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 --2- 172.21.15.181:0/2271776115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd14c0cb3b0 0x7fd14c150420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:23.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 >> 172.21.15.181:0/2271776115 conn(0x7fd14c0b1c90 msgr2=0x7fd14c1599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 shutdown_connections 2024-03-29T04:10:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:23.925+0000 7fd15403c640 1 -- 172.21.15.181:0/2271776115 wait complete. 2024-03-29T04:10:23.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T04:10:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.041+0000 7f0ade6e6640 1 Processor -- start 2024-03-29T04:10:24.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.041+0000 7f0ade6e6640 1 -- start start 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ade6e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ad80b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ade6e6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ad8072d50 0x7f0ad80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ade6e6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad80b35e0 con 0x7f0ad815f800 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ade6e6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad80b3e80 con 0x7f0ad8072d50 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ade6e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad80b38c0 con 0x7f0ad8144cb0 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ad77fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0ad815f800 0x7f0ad8072640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54534/0 (socket says 172.21.15.181:54534) 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ad77fe640 1 -- 172.21.15.181:0/3923679451 learned_addr learned my addr 172.21.15.181:0/3923679451 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3923679451 >> v2:172.21.15.187:3300/0 conn(0x7f0ad8072d50 0x7f0ad80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018463100 0 0) 0x7f0ad80b35e0 con 0x7f0ad815f800 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ad7fff640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ad80b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.042+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac0003520 con 0x7f0ad815f800 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1272155800 0 0) 0x7f0ac0003520 con 0x7f0ad815f800 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 msgr2=0x7f0ad80b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ad80b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 >> v2:172.21.15.187:3300/0 conn(0x7f0ad8072d50 msgr2=0x7f0ad80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 --2- 172.21.15.181:0/3923679451 >> v2:172.21.15.187:3300/0 conn(0x7f0ad8072d50 0x7f0ad80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad80b3ba0 con 0x7f0ad815f800 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f0acc001670 con 0x7f0ad815f800 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 >> v1:172.21.15.181:6789/0 conn(0x7f0ad815f800 legacy=0x7f0ad8072640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.025 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ac00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac0003f30 0x7f0ac0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ac0004d40 0x7f0ac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad80b3e80 con 0x7f0ac0004d40 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad80b38c0 con 0x7f0ac0003f30 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac000dbd0 con 0x7f0ad8144cb0 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ac0004d40 0x7f0ac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad77fe640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac0003f30 0x7f0ac0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0ad7fff640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ac00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 -- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 msgr2=0x7f0ac00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad8144cb0 0x7f0ac00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 -- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac0003f30 msgr2=0x7f0ac0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac0003f30 0x7f0ac0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.043+0000 7f0adcc83640 1 -- 172.21.15.181:0/3923679451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac0003520 con 0x7f0ac0004d40 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ac0004d40 0x7f0ac00051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009f90 tx=0x7f0ac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1d392d4ab840e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0039070 con 0x7f0ac0004d40 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad002cc00 con 0x7f0ac0004d40 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0ad6ffd640 1 -- 172.21.15.181:0/3923679451 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0009c10 con 0x7f0ac0004d40 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3923679451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ac0004d40 msgr2=0x7f0ac00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.044+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3923679451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ac0004d40 0x7f0ac00051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009f90 tx=0x7f0ac0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.045+0000 7f0adcc83640 1 -- 172.21.15.181:0/3923679451 reap_dead start 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.045+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3923679451 shutdown_connections 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.045+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3923679451 >> 172.21.15.181:0/3923679451 conn(0x7f0ad80b5b00 msgr2=0x7f0ad806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.045+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3923679451 shutdown_connections 2024-03-29T04:10:24.027 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.045+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3923679451 wait complete. 2024-03-29T04:10:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.046+0000 7f0ade6e6640 1 Processor -- start 2024-03-29T04:10:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.046+0000 7f0ade6e6640 1 -- start start 2024-03-29T04:10:24.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad815a9a0 0x7f0ad81434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 0x7f0ad8135500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad80b2f20 con 0x7f0ad8137130 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad815fc40 con 0x7f0ad815ad80 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad80b30a0 con 0x7f0ad815a9a0 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55028/0 (socket says 172.21.15.181:55028) 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 -- 172.21.15.181:0/3737619701 learned_addr learned my addr 172.21.15.181:0/3737619701 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad77fe640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 0x7f0ad8135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad7fff640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad815a9a0 0x7f0ad81434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 -- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad815a9a0 msgr2=0x7f0ad81434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad815a9a0 0x7f0ad81434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 -- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 msgr2=0x7f0ad8135500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 0x7f0ad8135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 -- 172.21.15.181:0/3737619701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad81391e0 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad77fe640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 0x7f0ad8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0adcc83640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002c6d0 tx=0x7f0ad0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0045070 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad003a040 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0039070 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad8140b80 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.047+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad8137980 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.048+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad8075890 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.049+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0ad0003bd0 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.049+0000 7f0ad4ff9640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 0x7f0ac406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.049+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0ad00ad0f0 con 0x7f0ad8137130 2024-03-29T04:10:24.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.049+0000 7f0ad7fff640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 0x7f0ac406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.050+0000 7f0ad7fff640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 0x7f0ac406e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009b00 tx=0x7f0ac8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.054+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0ad81391e0 con 0x7f0ad8137130 2024-03-29T04:10:24.247 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.268+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0ad80b4030 con 0x7f0ad8137130 2024-03-29T04:10:24.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.300+0000 7f0ad4ff9640 1 -- 172.21.15.181:0/3737619701 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f0ad007afc0 con 0x7f0ad8137130 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 msgr2=0x7f0ac406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 0x7f0ac406e9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009b00 tx=0x7f0ac8008040 comp rx=0 tx=0).stop 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 msgr2=0x7f0ad8135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0ad002c6d0 tx=0x7f0ad0009750 comp rx=0 tx=0).stop 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 shutdown_connections 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad8137130 0x7f0ad8135a40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ad815ad80 0x7f0ad8135500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f0ac406c530 0x7f0ac406e9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 --2- 172.21.15.181:0/3737619701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ad815a9a0 0x7f0ad81434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 >> 172.21.15.181:0/3737619701 conn(0x7f0ad80b5b00 msgr2=0x7f0ad813edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 shutdown_connections 2024-03-29T04:10:24.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.303+0000 7f0ade6e6640 1 -- 172.21.15.181:0/3737619701 wait complete. 2024-03-29T04:10:24.294 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T04:10:24.294 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T04:10:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.418+0000 7fc3a6f74640 1 Processor -- start 2024-03-29T04:10:24.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.418+0000 7fc3a6f74640 1 -- start start 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc3a6f74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc3a00b5dc0 0x7fc3a00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc3a6f74640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a00b66e0 con 0x7fc3a00ba090 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc3a6f74640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a00d2ff0 con 0x7fc3a00b73d0 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc3a6f74640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a00d2d90 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc3a4d10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc3a00b5dc0 0x7fc3a00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc39ffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc3a00ba090 0x7fc3a00d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54540/0 (socket says 172.21.15.181:54540) 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.419+0000 7fc39ffff640 1 -- 172.21.15.181:0/2050342628 learned_addr learned my addr 172.21.15.181:0/2050342628 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.420+0000 7fc3a4d10640 1 -- 172.21.15.181:0/2050342628 >> v1:172.21.15.187:6789/0 conn(0x7fc3a00b73d0 legacy=0x7fc3a00b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.420+0000 7fc3a4d10640 1 -- 172.21.15.181:0/2050342628 >> v1:172.21.15.181:6789/0 conn(0x7fc3a00ba090 legacy=0x7fc3a00d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.420+0000 7fc39f7fe640 1 -- 172.21.15.181:0/2050342628 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432906389 0 0) 0x7fc3a00b66e0 con 0x7fc3a00ba090 2024-03-29T04:10:24.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.420+0000 7fc3a4d10640 1 -- 172.21.15.181:0/2050342628 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a00d32d0 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.420+0000 7fc3a4d10640 1 --2- 172.21.15.181:0/2050342628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a00b61a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3940097a0 tx=0x7fc39402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d037186665b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc39f7fe640 1 -- 172.21.15.181:0/2050342628 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394034070 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc39f7fe640 1 -- 172.21.15.181:0/2050342628 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc394030610 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc39f7fe640 1 -- 172.21.15.181:0/2050342628 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3940309e0 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 -- 172.21.15.181:0/2050342628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00b5dc0 msgr2=0x7fc3a00b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/2050342628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a00b61a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3940097a0 tx=0x7fc39402b960 comp rx=0 tx=0).stop 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 -- 172.21.15.181:0/2050342628 shutdown_connections 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/2050342628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a00b61a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 -- 172.21.15.181:0/2050342628 >> 172.21.15.181:0/2050342628 conn(0x7fc3a00678f0 msgr2=0x7fc3a00d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:24.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 -- 172.21.15.181:0/2050342628 shutdown_connections 2024-03-29T04:10:24.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.421+0000 7fc3a6f74640 1 -- 172.21.15.181:0/2050342628 wait complete. 2024-03-29T04:10:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.422+0000 7fc3a6f74640 1 Processor -- start 2024-03-29T04:10:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.422+0000 7fc3a6f74640 1 -- start start 2024-03-29T04:10:24.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a01f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3a00b73d0 0x7fc3a00ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00d3b70 con 0x7fc3a00ba090 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00d3890 con 0x7fc3a00b5dc0 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a6f74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a00d4090 con 0x7fc3a00b73d0 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a5511640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a5511640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55056/0 (socket says 172.21.15.181:55056) 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.423+0000 7fc3a5511640 1 -- 172.21.15.181:0/1135891104 learned_addr learned my addr 172.21.15.181:0/1135891104 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a4d10640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a01f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc39ffff640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3a00b73d0 0x7fc3a00ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 -- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3a00b73d0 msgr2=0x7fc3a00ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3a00b73d0 0x7fc3a00ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 -- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 msgr2=0x7fc3a01f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a01f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 -- 172.21.15.181:0/1135891104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a0160050 con 0x7fc3a00ba090 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a4d10640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a01f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a5511640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c7d0 tx=0x7fc39800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398003b70 con 0x7fc3a00ba090 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc398003d30 con 0x7fc3a00ba090 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a0160330 con 0x7fc3a00ba090 2024-03-29T04:10:24.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39800aa90 con 0x7fc3a00ba090 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.424+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a01608f0 con 0x7fc3a00ba090 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.425+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a0160af0 con 0x7fc3a00ba090 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.426+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc39800ac30 con 0x7fc3a00ba090 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.426+0000 7fc39d7fa640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 0x7fc38406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.426+0000 7fc3a4d10640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 0x7fc38406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.426+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc39808b790 con 0x7fc3a00ba090 2024-03-29T04:10:24.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.427+0000 7fc3a4d10640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 0x7fc38406e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc394036000 tx=0x7fc39403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:24.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.430+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3a0160af0 con 0x7fc3a00ba090 2024-03-29T04:10:24.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:24.645+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc3a01f4550 con 0x7fc3a00ba090 2024-03-29T04:10:25.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.305+0000 7fc39d7fa640 1 -- 172.21.15.181:0/1135891104 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fc398059c20 con 0x7fc3a00ba090 2024-03-29T04:10:25.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 msgr2=0x7fc38406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 0x7fc38406e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc394036000 tx=0x7fc39403b040 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 msgr2=0x7fc3a015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c7d0 tx=0x7fc39800cc90 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 shutdown_connections 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3a00ba090 0x7fc3a015a180 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc38406c530 0x7fc38406e9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3a00b73d0 0x7fc3a00ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 --2- 172.21.15.181:0/1135891104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3a00b5dc0 0x7fc3a01f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 >> 172.21.15.181:0/1135891104 conn(0x7fc3a00678f0 msgr2=0x7fc3a00b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 shutdown_connections 2024-03-29T04:10:25.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.308+0000 7fc3a6f74640 1 -- 172.21.15.181:0/1135891104 wait complete. 2024-03-29T04:10:25.298 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T04:10:25.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.425+0000 7fb1416a5640 1 Processor -- start 2024-03-29T04:10:25.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.425+0000 7fb1416a5640 1 -- start start 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.426+0000 7fb1416a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb13c071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.426+0000 7fb1416a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13c141910 con 0x7fb13c0b5950 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.426+0000 7fb1416a5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13c071e60 con 0x7fb13c162030 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.426+0000 7fb1416a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c162af0 con 0x7fb13c141530 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.426+0000 7fb13b7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb13c0b5950 0x7fb13c141c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54548/0 (socket says 172.21.15.181:54548) 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb13b7fe640 1 -- 172.21.15.181:0/2714260468 learned_addr learned my addr 172.21.15.181:0/2714260468 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb13affd640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb13c071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454831557 0 0) 0x7fb13c141910 con 0x7fb13c0b5950 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb124003520 con 0x7fb13c0b5950 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990367775 0 0) 0x7fb13c071e60 con 0x7fb13c162030 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb13c141910 con 0x7fb13c162030 2024-03-29T04:10:25.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4156164454 0 0) 0x7fb124003520 con 0x7fb13c0b5950 2024-03-29T04:10:25.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 msgr2=0x7fb13c071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb13c071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 >> v1:172.21.15.187:6789/0 conn(0x7fb13c162030 legacy=0x7fb13c0b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c162dd0 con 0x7fb13c0b5950 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fb134001a10 con 0x7fb13c0b5950 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 >> v1:172.21.15.181:6789/0 conn(0x7fb13c0b5950 legacy=0x7fb13c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb124003b70 0x7fb124003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb124004800 0x7fb124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb124005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c162af0 con 0x7fb124004800 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb12400bb50 con 0x7fb124003b70 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb12400bd20 con 0x7fb13c141530 2024-03-29T04:10:25.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb124004800 0x7fb124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb13affd640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb124003b70 0x7fb124003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.427+0000 7fb13b7fe640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb124005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 -- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 msgr2=0x7fb124005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb13c141530 0x7fb124005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 -- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb124003b70 msgr2=0x7fb124003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb124003b70 0x7fb124003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 -- 172.21.15.181:0/2714260468 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb124003520 con 0x7fb124004800 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb124004800 0x7fb124004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb130001900 tx=0x7fb124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3675534ac78a2fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130037070 con 0x7fb124004800 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1300052d0 con 0x7fb124004800 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.428+0000 7fb139ffb640 1 -- 172.21.15.181:0/2714260468 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300014f0 con 0x7fb124004800 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 -- 172.21.15.181:0/2714260468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb124004800 msgr2=0x7fb124004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 --2- 172.21.15.181:0/2714260468 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb124004800 0x7fb124004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb130001900 tx=0x7fb124005830 comp rx=0 tx=0).stop 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb13b7fe640 1 -- 172.21.15.181:0/2714260468 reap_dead start 2024-03-29T04:10:25.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 -- 172.21.15.181:0/2714260468 shutdown_connections 2024-03-29T04:10:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 -- 172.21.15.181:0/2714260468 >> 172.21.15.181:0/2714260468 conn(0x7fb13c06f8b0 msgr2=0x7fb13c06fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 -- 172.21.15.181:0/2714260468 shutdown_connections 2024-03-29T04:10:25.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.429+0000 7fb1416a5640 1 -- 172.21.15.181:0/2714260468 wait complete. 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.430+0000 7fb1416a5640 1 Processor -- start 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.430+0000 7fb1416a5640 1 -- start start 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb124004800 0x7fb13c1338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb13c15e520 0x7fb13c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c0b1950 con 0x7fb13c15e0a0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c071400 con 0x7fb13c15e520 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb1416a5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c071920 con 0x7fb124004800 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb13a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb13a7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55076/0 (socket says 172.21.15.181:55076) 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb13a7fc640 1 -- 172.21.15.181:0/285020762 learned_addr learned my addr 172.21.15.181:0/285020762 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb13b7fe640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb13c15e520 0x7fb13c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.431+0000 7fb13affd640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb124004800 0x7fb13c1338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 -- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb124004800 msgr2=0x7fb13c1338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb124004800 0x7fb13c1338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 -- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb13c15e520 msgr2=0x7fb13c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb13c15e520 0x7fb13c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 -- 172.21.15.181:0/285020762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c1407f0 con 0x7fb13c15e0a0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb13a7fc640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb130035040 tx=0x7fb1300012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130042020 con 0x7fb13c15e0a0 2024-03-29T04:10:25.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb130038040 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c133e30 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130037070 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.432+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c15c2d0 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.433+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c075a10 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.433+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb130045020 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.434+0000 7fb1237fe640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 0x7fb12806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.434+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb1300ad2f0 con 0x7fb13c15e0a0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.434+0000 7fb13affd640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 0x7fb12806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.434+0000 7fb13affd640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 0x7fb12806e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009a60 tx=0x7fb12c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.438+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb13004c030 con 0x7fb13c15e0a0 2024-03-29T04:10:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.657+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb13c0d2f00 con 0x7fb13c15e0a0 2024-03-29T04:10:25.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.658+0000 7fb1237fe640 1 -- 172.21.15.181:0/285020762 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fb13007b0a0 con 0x7fb13c15e0a0 2024-03-29T04:10:25.637 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:25.638 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:24.646619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:10:25.638 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2024-03-29T04:10:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 msgr2=0x7fb12806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 0x7fb12806e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009a60 tx=0x7fb12c009190 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 msgr2=0x7fb13c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb130035040 tx=0x7fb1300012d0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 shutdown_connections 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb13c15e520 0x7fb13c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb13c15e0a0 0x7fb13c15b410 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb12806c530 0x7fb12806e9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 --2- 172.21.15.181:0/285020762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb124004800 0x7fb13c1338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 >> 172.21.15.181:0/285020762 conn(0x7fb13c06f8b0 msgr2=0x7fb13c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 shutdown_connections 2024-03-29T04:10:25.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.660+0000 7fb1416a5640 1 -- 172.21.15.181:0/285020762 wait complete. 2024-03-29T04:10:25.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:10:25.750 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.771+0000 7f83a9af2640 1 Processor -- start 2024-03-29T04:10:25.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.771+0000 7f83a9af2640 1 -- start start 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a9af2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83a40d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a9af2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40d36f0 con 0x7f83a40d30d0 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a9af2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40d39d0 con 0x7f83a40d48c0 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a9af2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f83a40d3cb0 con 0x7f83a415e950 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a3fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f83a40d30d0 0x7f83a40b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54554/0 (socket says 172.21.15.181:54554) 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a3fff640 1 -- 172.21.15.181:0/546078993 learned_addr learned my addr 172.21.15.181:0/546078993 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:25.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a37fe640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83a40d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003094418 0 0) 0x7f83a40d36f0 con 0x7f83a40d30d0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8390003520 con 0x7f83a40d30d0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 826856740 0 0) 0x7f8390003520 con 0x7f83a40d30d0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 msgr2=0x7f83a40d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83a40d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.772+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 >> v1:172.21.15.187:6789/0 conn(0x7f83a40d48c0 legacy=0x7f83a40d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a40b7ac0 con 0x7f83a40d30d0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f839c0019f0 con 0x7f83a40d30d0 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 >> v1:172.21.15.181:6789/0 conn(0x7f83a40d30d0 legacy=0x7f83a40b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f8390004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8390004d40 0x7f83900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40d3cb0 con 0x7f8390003f20 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f839000ba20 con 0x7f8390004d40 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f839000bc20 con 0x7f83a415e950 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f8390004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a37fe640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f83a3fff640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8390004d40 0x7f83900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 -- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 msgr2=0x7f83900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a415e950 0x7f83900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 -- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8390004d40 msgr2=0x7f83900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8390004d40 0x7f83900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.773+0000 7f839affd640 1 -- 172.21.15.181:0/546078993 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8390003520 con 0x7f8390003f20 2024-03-29T04:10:25.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f839affd640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f8390004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f838c001860 tx=0x7f8390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21838f95028165bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c001430 con 0x7f8390003f20 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c032620 con 0x7f8390003f20 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a2ffd640 1 -- 172.21.15.181:0/546078993 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c032a10 con 0x7f8390003f20 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 -- 172.21.15.181:0/546078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 msgr2=0x7f8390004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 --2- 172.21.15.181:0/546078993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f8390004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f838c001860 tx=0x7f8390005830 comp rx=0 tx=0).stop 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a3fff640 1 -- 172.21.15.181:0/546078993 reap_dead start 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 -- 172.21.15.181:0/546078993 shutdown_connections 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 -- 172.21.15.181:0/546078993 >> 172.21.15.181:0/546078993 conn(0x7f83a40676e0 msgr2=0x7f83a4067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 -- 172.21.15.181:0/546078993 shutdown_connections 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.774+0000 7f83a9af2640 1 -- 172.21.15.181:0/546078993 wait complete. 2024-03-29T04:10:25.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.776+0000 7f83a9af2640 1 Processor -- start 2024-03-29T04:10:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.776+0000 7f83a9af2640 1 -- start start 2024-03-29T04:10:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a4161c60 0x7f83a41f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 0x7f83a40ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83a415ed30 con 0x7f8390003f20 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40d4d00 con 0x7f83a40ca040 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a9af2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f83a415eeb0 con 0x7f83a4161c60 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55096/0 (socket says 172.21.15.181:55096) 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 -- 172.21.15.181:0/2015399769 learned_addr learned my addr 172.21.15.181:0/2015399769 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a3fff640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 0x7f83a40ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f839affd640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a4161c60 0x7f83a41f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 -- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a4161c60 msgr2=0x7f83a41f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a4161c60 0x7f83a41f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 -- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 msgr2=0x7f83a40ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 0x7f83a40ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:25.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 -- 172.21.15.181:0/2015399769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a40c8470 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a3fff640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 0x7f83a40ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.777+0000 7f83a37fe640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83940119a0 tx=0x7f8394011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.778+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400faf0 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.778+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839400fc90 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.778+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394014440 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.778+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41f5110 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.778+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a40c89c0 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.779+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a40c81c0 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.779+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8394012040 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.780+0000 7f83a0ff9640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 0x7f838406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.780+0000 7f839affd640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 0x7f838406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.780+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f839408bee0 con 0x7f8390003f20 2024-03-29T04:10:25.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.780+0000 7f839affd640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 0x7f838406ea20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f838c035000 tx=0x7f838c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:25.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:25.784+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f83a41f5110 con 0x7f8390003f20 2024-03-29T04:10:26.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.088+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83a4075a10 con 0x7f8390003f20 2024-03-29T04:10:26.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.089+0000 7f83a0ff9640 1 -- 172.21.15.181:0/2015399769 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16954 (secure 0 0 0) 0x7f8394060030 con 0x7f8390003f20 2024-03-29T04:10:26.068 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:26.069 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":39,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:24.297404+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:10:26.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 msgr2=0x7f838406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 0x7f838406ea20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f838c035000 tx=0x7f838c030000 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 msgr2=0x7f83a415d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83940119a0 tx=0x7f8394011e60 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 shutdown_connections 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83a40ca040 0x7f83a40ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f838406c560 0x7f838406ea20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83a4161c60 0x7f83a41f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 --2- 172.21.15.181:0/2015399769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8390003f20 0x7f83a415d010 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 >> 172.21.15.181:0/2015399769 conn(0x7f83a40676e0 msgr2=0x7f83a40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 shutdown_connections 2024-03-29T04:10:26.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.091+0000 7f83a9af2640 1 -- 172.21.15.181:0/2015399769 wait complete. 2024-03-29T04:10:26.082 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T04:10:26.082 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.204+0000 7f3d91274640 1 Processor -- start 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d91274640 1 -- start start 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d91274640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d8c0d4a90 0x7f3d8c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d91274640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8c0d36f0 con 0x7f3d8c15e950 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d91274640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8c0d39d0 con 0x7f3d8c0d2f00 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d91274640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c0d3cb0 con 0x7f3d8c0d4a90 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d8affd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3d8c15e950 0x7f3d8c0d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54562/0 (socket says 172.21.15.181:54562) 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.205+0000 7f3d8affd640 1 -- 172.21.15.181:0/597098108 learned_addr learned my addr 172.21.15.181:0/597098108 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8a7fc640 1 --2- 172.21.15.181:0/597098108 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c0d4a90 0x7f3d8c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156194095 0 0) 0x7f3d8c0d36f0 con 0x7f3d8c15e950 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d74003520 con 0x7f3d8c15e950 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 584480042 0 0) 0x7f3d74003520 con 0x7f3d8c15e950 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c0d4a90 msgr2=0x7f3d8c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 --2- 172.21.15.181:0/597098108 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c0d4a90 0x7f3d8c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 >> v1:172.21.15.187:6789/0 conn(0x7f3d8c0d2f00 legacy=0x7f3d8c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c0b7ac0 con 0x7f3d8c15e950 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f3d7c001670 con 0x7f3d8c15e950 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 >> v1:172.21.15.181:6789/0 conn(0x7f3d8c15e950 legacy=0x7f3d8c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d740039b0 0x7f3d74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d740043b0 0x7f3d74004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c0d3cb0 con 0x7f3d74005080 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7400bf30 con 0x7f3d740039b0 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7400c130 con 0x7f3d740043b0 2024-03-29T04:10:26.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8affd640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d740039b0 0x7f3d74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8a7fc640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d740043b0 0x7f3d74004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 -- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d740043b0 msgr2=0x7f3d74004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d740043b0 0x7f3d74004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 -- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d740039b0 msgr2=0x7f3d74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d740039b0 0x7f3d74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.206+0000 7f3d8b7fe640 1 -- 172.21.15.181:0/597098108 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d74003520 con 0x7f3d74005080 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.207+0000 7f3d8b7fe640 1 -- 172.21.15.181:0/597098108 reap_dead start 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.207+0000 7f3d8b7fe640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d7400b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d84002010 tx=0x7f3d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b441ee29d7f11c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.207+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84001be0 con 0x7f3d74005080 2024-03-29T04:10:26.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.207+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d84003b70 con 0x7f3d74005080 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.207+0000 7f3d89ffb640 1 -- 172.21.15.181:0/597098108 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84005110 con 0x7f3d74005080 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 -- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 msgr2=0x7f3d7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d7400b9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d84002010 tx=0x7f3d74005830 comp rx=0 tx=0).stop 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 -- 172.21.15.181:0/597098108 shutdown_connections 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 --2- 172.21.15.181:0/597098108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d7400b9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 -- 172.21.15.181:0/597098108 >> 172.21.15.181:0/597098108 conn(0x7f3d8c0676e0 msgr2=0x7f3d8c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 -- 172.21.15.181:0/597098108 shutdown_connections 2024-03-29T04:10:26.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.208+0000 7f3d91274640 1 -- 172.21.15.181:0/597098108 wait complete. 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.209+0000 7f3d91274640 1 Processor -- start 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.209+0000 7f3d91274640 1 -- start start 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c0cd960 0x7f3d8c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c15ace0 0x7f3d8c160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c15ed30 con 0x7f3d74005080 2024-03-29T04:10:26.194 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c0d33b0 con 0x7f3d8c15ace0 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d91274640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c15eeb0 con 0x7f3d8c0cd960 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55120/0 (socket says 172.21.15.181:55120) 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 -- 172.21.15.181:0/1500898318 learned_addr learned my addr 172.21.15.181:0/1500898318 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8a7fc640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c0cd960 0x7f3d8c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8b7fe640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c15ace0 0x7f3d8c160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 -- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c0cd960 msgr2=0x7f3d8c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c0cd960 0x7f3d8c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 -- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c15ace0 msgr2=0x7f3d8c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c15ace0 0x7f3d8c160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.210+0000 7f3d8affd640 1 -- 172.21.15.181:0/1500898318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c0cc620 con 0x7f3d74005080 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d8affd640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007290 tx=0x7f3d7c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c001650 con 0x7f3d74005080 2024-03-29T04:10:26.195 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c018070 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00fb60 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c0cc8c0 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.211+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c0cce80 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.212+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c075a10 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.212+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3d7c00fd00 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.213+0000 7f3d737fe640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 0x7f3d7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.213+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3d7c08b850 con 0x7f3d74005080 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.213+0000 7f3d8a7fc640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 0x7f3d7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.213+0000 7f3d8a7fc640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 0x7f3d7806e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c1557f0 tx=0x7f3d80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.196 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.217+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3d7c059ce0 con 0x7f3d74005080 2024-03-29T04:10:26.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.460+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d8c06b580 con 0x7f3d7806c530 2024-03-29T04:10:26.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.466+0000 7f3d737fe640 1 -- 172.21.15.181:0/1500898318 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250415 (secure 0 0 0) 0x7f3d8c06b580 con 0x7f3d7806c530 2024-03-29T04:10:26.448 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:26.450 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 msgr2=0x7f3d7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 0x7f3d7806e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c1557f0 tx=0x7f3d80008040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 msgr2=0x7f3d8c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007290 tx=0x7f3d7c007750 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 shutdown_connections 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c15ace0 0x7f3d8c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3d7806c530 0x7f3d7806e9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c0cd960 0x7f3d8c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 --2- 172.21.15.181:0/1500898318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d74005080 0x7f3d8c0ca1e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 >> 172.21.15.181:0/1500898318 conn(0x7f3d8c0676e0 msgr2=0x7f3d8c15c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 shutdown_connections 2024-03-29T04:10:26.452 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.469+0000 7f3d91274640 1 -- 172.21.15.181:0/1500898318 wait complete. 2024-03-29T04:10:26.460 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-03-29T04:10:25.587035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220684,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754036,"statfs":{"total":773094113280,"available":772868132864,"internally_reserved":0,"allocated":4616192,"data_stored":2649928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.305977"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318073+0000","last_change":"2024-03-29T04:10:23.313049+0000","last_active":"2024-03-29T04:10:23.318073+0000","last_peered":"2024-03-29T04:10:23.318073+0000","last_clean":"2024-03-29T04:10:23.318073+0000","last_became_active":"2024-03-29T04:10:23.312619+0000","last_became_peered":"2024-03-29T04:10:23.312619+0000","last_unstale":"2024-03-29T04:10:23.318073+0000","last_undegraded":"2024-03-29T04:10:23.318073+0000","last_fullsized":"2024-03-29T04:10:23.318073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309796+0000","last_change":"2024-03-29T04:10:23.304958+0000","last_active":"2024-03-29T04:10:23.309796+0000","last_peered":"2024-03-29T04:10:23.309796+0000","last_clean":"2024-03-29T04:10:23.309796+0000","last_became_active":"2024-03-29T04:10:23.304714+0000","last_became_peered":"2024-03-29T04:10:23.304714+0000","last_unstale":"2024-03-29T04:10:23.309796+0000","last_undegraded":"2024-03-29T04:10:23.309796+0000","last_fullsized":"2024-03-29T04:10:23.309796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322299+0000","last_change":"2024-03-29T04:10:23.315527+0000","last_active":"2024-03-29T04:10:23.322299+0000","last_peered":"2024-03-29T04:10:23.322299+0000","last_clean":"2024-03-29T04:10:23.322299+0000","last_became_active":"2024-03-29T04:10:23.315242+0000","last_became_peered":"2024-03-29T04:10:23.315242+0000","last_unstale":"2024-03-29T04:10:23.322299+0000","last_undegraded":"2024-03-29T04:10:23.322299+0000","last_fullsized":"2024-03-29T04:10:23.322299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321235+0000","last_change":"2024-03-29T04:10:23.315865+0000","last_active":"2024-03-29T04:10:23.321235+0000","last_peered":"2024-03-29T04:10:23.321235+0000","last_clean":"2024-03-29T04:10:23.321235+0000","last_became_active":"2024-03-29T04:10:23.315580+0000","last_became_peered":"2024-03-29T04:10:23.315580+0000","last_unstale":"2024-03-29T04:10:23.321235+0000","last_undegraded":"2024-03-29T04:10:23.321235+0000","last_fullsized":"2024-03-29T04:10:23.321235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:33:01.952778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321146+0000","last_change":"2024-03-29T04:10:23.315150+0000","last_active":"2024-03-29T04:10:23.321146+0000","last_peered":"2024-03-29T04:10:23.321146+0000","last_clean":"2024-03-29T04:10:23.321146+0000","last_became_active":"2024-03-29T04:10:23.314879+0000","last_became_peered":"2024-03-29T04:10:23.314879+0000","last_unstale":"2024-03-29T04:10:23.321146+0000","last_undegraded":"2024-03-29T04:10:23.321146+0000","last_fullsized":"2024-03-29T04:10:23.321146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.011645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316771+0000","last_change":"2024-03-29T04:10:23.311220+0000","last_active":"2024-03-29T04:10:23.316771+0000","last_peered":"2024-03-29T04:10:23.316771+0000","last_clean":"2024-03-29T04:10:23.316771+0000","last_became_active":"2024-03-29T04:10:23.310727+0000","last_became_peered":"2024-03-29T04:10:23.310727+0000","last_unstale":"2024-03-29T04:10:23.316771+0000","last_undegraded":"2024-03-29T04:10:23.316771+0000","last_fullsized":"2024-03-29T04:10:23.316771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.302395+0000","last_change":"2024-03-29T04:10:22.302395+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.302395+0000","last_undegraded":"2024-03-29T04:10:22.302395+0000","last_fullsized":"2024-03-29T04:10:22.302395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317690+0000","last_change":"2024-03-29T04:10:23.314124+0000","last_active":"2024-03-29T04:10:23.317690+0000","last_peered":"2024-03-29T04:10:23.317690+0000","last_clean":"2024-03-29T04:10:23.317690+0000","last_became_active":"2024-03-29T04:10:23.313434+0000","last_became_peered":"2024-03-29T04:10:23.313434+0000","last_unstale":"2024-03-29T04:10:23.317690+0000","last_undegraded":"2024-03-29T04:10:23.317690+0000","last_fullsized":"2024-03-29T04:10:23.317690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322272+0000","last_change":"2024-03-29T04:10:23.315461+0000","last_active":"2024-03-29T04:10:23.322272+0000","last_peered":"2024-03-29T04:10:23.322272+0000","last_clean":"2024-03-29T04:10:23.322272+0000","last_became_active":"2024-03-29T04:10:23.315163+0000","last_became_peered":"2024-03-29T04:10:23.315163+0000","last_unstale":"2024-03-29T04:10:23.322272+0000","last_undegraded":"2024-03-29T04:10:23.322272+0000","last_fullsized":"2024-03-29T04:10:23.322272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.303882+0000","last_change":"2024-03-29T04:10:22.303882+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.303882+0000","last_undegraded":"2024-03-29T04:10:22.303882+0000","last_fullsized":"2024-03-29T04:10:22.303882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301429+0000","last_change":"2024-03-29T04:10:22.301429+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301429+0000","last_undegraded":"2024-03-29T04:10:22.301429+0000","last_fullsized":"2024-03-29T04:10:22.301429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311095+0000","last_change":"2024-03-29T04:10:23.309450+0000","last_active":"2024-03-29T04:10:23.311095+0000","last_peered":"2024-03-29T04:10:23.311095+0000","last_clean":"2024-03-29T04:10:23.311095+0000","last_became_active":"2024-03-29T04:10:23.309186+0000","last_became_peered":"2024-03-29T04:10:23.309186+0000","last_unstale":"2024-03-29T04:10:23.311095+0000","last_undegraded":"2024-03-29T04:10:23.311095+0000","last_fullsized":"2024-03-29T04:10:23.311095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.135609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300951+0000","last_change":"2024-03-29T04:10:22.300951+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300951+0000","last_undegraded":"2024-03-29T04:10:22.300951+0000","last_fullsized":"2024-03-29T04:10:22.300951+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.293775+0000","last_change":"2024-03-29T04:10:22.293775+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.293775+0000","last_undegraded":"2024-03-29T04:10:22.293775+0000","last_fullsized":"2024-03-29T04:10:22.293775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301942+0000","last_change":"2024-03-29T04:10:22.301942+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301942+0000","last_undegraded":"2024-03-29T04:10:22.301942+0000","last_fullsized":"2024-03-29T04:10:22.301942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316831+0000","last_change":"2024-03-29T04:10:23.311958+0000","last_active":"2024-03-29T04:10:23.316831+0000","last_peered":"2024-03-29T04:10:23.316831+0000","last_clean":"2024-03-29T04:10:23.316831+0000","last_became_active":"2024-03-29T04:10:23.311700+0000","last_became_peered":"2024-03-29T04:10:23.311700+0000","last_unstale":"2024-03-29T04:10:23.316831+0000","last_undegraded":"2024-03-29T04:10:23.316831+0000","last_fullsized":"2024-03-29T04:10:23.316831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316930+0000","last_change":"2024-03-29T04:10:23.313386+0000","last_active":"2024-03-29T04:10:23.316930+0000","last_peered":"2024-03-29T04:10:23.316930+0000","last_clean":"2024-03-29T04:10:23.316930+0000","last_became_active":"2024-03-29T04:10:23.313083+0000","last_became_peered":"2024-03-29T04:10:23.313083+0000","last_unstale":"2024-03-29T04:10:23.316930+0000","last_undegraded":"2024-03-29T04:10:23.316930+0000","last_fullsized":"2024-03-29T04:10:23.316930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304763+0000","last_change":"2024-03-29T04:10:23.300122+0000","last_active":"2024-03-29T04:10:23.304763+0000","last_peered":"2024-03-29T04:10:23.304763+0000","last_clean":"2024-03-29T04:10:23.304763+0000","last_became_active":"2024-03-29T04:10:23.299864+0000","last_became_peered":"2024-03-29T04:10:23.299864+0000","last_unstale":"2024-03-29T04:10:23.304763+0000","last_undegraded":"2024-03-29T04:10:23.304763+0000","last_fullsized":"2024-03-29T04:10:23.304763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:29.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321486+0000","last_change":"2024-03-29T04:10:23.316162+0000","last_active":"2024-03-29T04:10:23.321486+0000","last_peered":"2024-03-29T04:10:23.321486+0000","last_clean":"2024-03-29T04:10:23.321486+0000","last_became_active":"2024-03-29T04:10:23.315894+0000","last_became_peered":"2024-03-29T04:10:23.315894+0000","last_unstale":"2024-03-29T04:10:23.321486+0000","last_undegraded":"2024-03-29T04:10:23.321486+0000","last_fullsized":"2024-03-29T04:10:23.321486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:37.304758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.305943+0000","last_change":"2024-03-29T04:10:23.297888+0000","last_active":"2024-03-29T04:10:23.305943+0000","last_peered":"2024-03-29T04:10:23.305943+0000","last_clean":"2024-03-29T04:10:23.305943+0000","last_became_active":"2024-03-29T04:10:23.297611+0000","last_became_peered":"2024-03-29T04:10:23.297611+0000","last_unstale":"2024-03-29T04:10:23.305943+0000","last_undegraded":"2024-03-29T04:10:23.305943+0000","last_fullsized":"2024-03-29T04:10:23.305943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:51:35.316362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300543+0000","last_change":"2024-03-29T04:10:22.300543+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300543+0000","last_undegraded":"2024-03-29T04:10:22.300543+0000","last_fullsized":"2024-03-29T04:10:22.300543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291376+0000","last_change":"2024-03-29T04:10:22.291376+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291376+0000","last_undegraded":"2024-03-29T04:10:22.291376+0000","last_fullsized":"2024-03-29T04:10:22.291376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317436+0000","last_change":"2024-03-29T04:10:23.315012+0000","last_active":"2024-03-29T04:10:23.317436+0000","last_peered":"2024-03-29T04:10:23.317436+0000","last_clean":"2024-03-29T04:10:23.317436+0000","last_became_active":"2024-03-29T04:10:23.312553+0000","last_became_peered":"2024-03-29T04:10:23.312553+0000","last_unstale":"2024-03-29T04:10:23.317436+0000","last_undegraded":"2024-03-29T04:10:23.317436+0000","last_fullsized":"2024-03-29T04:10:23.317436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:36.093598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322166+0000","last_change":"2024-03-29T04:10:23.318761+0000","last_active":"2024-03-29T04:10:23.322166+0000","last_peered":"2024-03-29T04:10:23.322166+0000","last_clean":"2024-03-29T04:10:23.322166+0000","last_became_active":"2024-03-29T04:10:23.318414+0000","last_became_peered":"2024-03-29T04:10:23.318414+0000","last_unstale":"2024-03-29T04:10:23.322166+0000","last_undegraded":"2024-03-29T04:10:23.322166+0000","last_fullsized":"2024-03-29T04:10:23.322166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:40.265277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319387+0000","last_change":"2024-03-29T04:10:23.312068+0000","last_active":"2024-03-29T04:10:23.319387+0000","last_peered":"2024-03-29T04:10:23.319387+0000","last_clean":"2024-03-29T04:10:23.319387+0000","last_became_active":"2024-03-29T04:10:23.311776+0000","last_became_peered":"2024-03-29T04:10:23.311776+0000","last_unstale":"2024-03-29T04:10:23.319387+0000","last_undegraded":"2024-03-29T04:10:23.319387+0000","last_fullsized":"2024-03-29T04:10:23.319387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:59.921229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320160+0000","last_change":"2024-03-29T04:10:23.318246+0000","last_active":"2024-03-29T04:10:23.320160+0000","last_peered":"2024-03-29T04:10:23.320160+0000","last_clean":"2024-03-29T04:10:23.320160+0000","last_became_active":"2024-03-29T04:10:23.317989+0000","last_became_peered":"2024-03-29T04:10:23.317989+0000","last_unstale":"2024-03-29T04:10:23.320160+0000","last_undegraded":"2024-03-29T04:10:23.320160+0000","last_fullsized":"2024-03-29T04:10:23.320160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309790+0000","last_change":"2024-03-29T04:10:23.308341+0000","last_active":"2024-03-29T04:10:23.309790+0000","last_peered":"2024-03-29T04:10:23.309790+0000","last_clean":"2024-03-29T04:10:23.309790+0000","last_became_active":"2024-03-29T04:10:23.307789+0000","last_became_peered":"2024-03-29T04:10:23.307789+0000","last_unstale":"2024-03-29T04:10:23.309790+0000","last_undegraded":"2024-03-29T04:10:23.309790+0000","last_fullsized":"2024-03-29T04:10:23.309790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:08.599452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.299555+0000","last_change":"2024-03-29T04:10:22.299555+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.299555+0000","last_undegraded":"2024-03-29T04:10:22.299555+0000","last_fullsized":"2024-03-29T04:10:22.299555+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308971+0000","last_change":"2024-03-29T04:10:23.307876+0000","last_active":"2024-03-29T04:10:23.308971+0000","last_peered":"2024-03-29T04:10:23.308971+0000","last_clean":"2024-03-29T04:10:23.308971+0000","last_became_active":"2024-03-29T04:10:23.307565+0000","last_became_peered":"2024-03-29T04:10:23.307565+0000","last_unstale":"2024-03-29T04:10:23.308971+0000","last_undegraded":"2024-03-29T04:10:23.308971+0000","last_fullsized":"2024-03-29T04:10:23.308971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:11.709686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308760+0000","last_change":"2024-03-29T04:10:23.306970+0000","last_active":"2024-03-29T04:10:23.308760+0000","last_peered":"2024-03-29T04:10:23.308760+0000","last_clean":"2024-03-29T04:10:23.308760+0000","last_became_active":"2024-03-29T04:10:23.306708+0000","last_became_peered":"2024-03-29T04:10:23.306708+0000","last_unstale":"2024-03-29T04:10:23.308760+0000","last_undegraded":"2024-03-29T04:10:23.308760+0000","last_fullsized":"2024-03-29T04:10:23.308760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:15.611882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317417+0000","last_change":"2024-03-29T04:10:23.314775+0000","last_active":"2024-03-29T04:10:23.317417+0000","last_peered":"2024-03-29T04:10:23.317417+0000","last_clean":"2024-03-29T04:10:23.317417+0000","last_became_active":"2024-03-29T04:10:23.312160+0000","last_became_peered":"2024-03-29T04:10:23.312160+0000","last_unstale":"2024-03-29T04:10:23.317417+0000","last_undegraded":"2024-03-29T04:10:23.317417+0000","last_fullsized":"2024-03-29T04:10:23.317417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.313388+0000","last_change":"2024-03-29T04:10:23.304939+0000","last_active":"2024-03-29T04:10:23.313388+0000","last_peered":"2024-03-29T04:10:23.313388+0000","last_clean":"2024-03-29T04:10:23.313388+0000","last_became_active":"2024-03-29T04:10:23.304682+0000","last_became_peered":"2024-03-29T04:10:23.304682+0000","last_unstale":"2024-03-29T04:10:23.313388+0000","last_undegraded":"2024-03-29T04:10:23.313388+0000","last_fullsized":"2024-03-29T04:10:23.313388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323623+0000","last_change":"2024-03-29T04:10:23.318116+0000","last_active":"2024-03-29T04:10:23.323623+0000","last_peered":"2024-03-29T04:10:23.323623+0000","last_clean":"2024-03-29T04:10:23.323623+0000","last_became_active":"2024-03-29T04:10:23.317859+0000","last_became_peered":"2024-03-29T04:10:23.317859+0000","last_unstale":"2024-03-29T04:10:23.323623+0000","last_undegraded":"2024-03-29T04:10:23.323623+0000","last_fullsized":"2024-03-29T04:10:23.323623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319317+0000","last_change":"2024-03-29T04:10:22.297429+0000","last_active":"2024-03-29T04:10:23.319317+0000","last_peered":"2024-03-29T04:10:23.319317+0000","last_clean":"2024-03-29T04:10:23.319317+0000","last_became_active":"2024-03-29T04:10:22.297077+0000","last_became_peered":"2024-03-29T04:10:22.297077+0000","last_unstale":"2024-03-29T04:10:23.319317+0000","last_undegraded":"2024-03-29T04:10:23.319317+0000","last_fullsized":"2024-03-29T04:10:23.319317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:10.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320194+0000","last_change":"2024-03-29T04:10:23.316949+0000","last_active":"2024-03-29T04:10:23.320194+0000","last_peered":"2024-03-29T04:10:23.320194+0000","last_clean":"2024-03-29T04:10:23.320194+0000","last_became_active":"2024-03-29T04:10:23.316665+0000","last_became_peered":"2024-03-29T04:10:23.316665+0000","last_unstale":"2024-03-29T04:10:23.320194+0000","last_undegraded":"2024-03-29T04:10:23.320194+0000","last_fullsized":"2024-03-29T04:10:23.320194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316490+0000","last_change":"2024-03-29T04:10:22.307974+0000","last_active":"2024-03-29T04:10:23.316490+0000","last_peered":"2024-03-29T04:10:23.316490+0000","last_clean":"2024-03-29T04:10:23.316490+0000","last_became_active":"2024-03-29T04:10:22.306547+0000","last_became_peered":"2024-03-29T04:10:22.306547+0000","last_unstale":"2024-03-29T04:10:23.316490+0000","last_undegraded":"2024-03-29T04:10:23.316490+0000","last_fullsized":"2024-03-29T04:10:23.316490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291935+0000","last_change":"2024-03-29T04:10:22.291935+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291935+0000","last_undegraded":"2024-03-29T04:10:22.291935+0000","last_fullsized":"2024-03-29T04:10:22.291935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319454+0000","last_change":"2024-03-29T04:10:22.301842+0000","last_active":"2024-03-29T04:10:23.319454+0000","last_peered":"2024-03-29T04:10:23.319454+0000","last_clean":"2024-03-29T04:10:23.319454+0000","last_became_active":"2024-03-29T04:10:22.301559+0000","last_became_peered":"2024-03-29T04:10:22.301559+0000","last_unstale":"2024-03-29T04:10:23.319454+0000","last_undegraded":"2024-03-29T04:10:23.319454+0000","last_fullsized":"2024-03-29T04:10:23.319454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311930+0000","last_change":"2024-03-29T04:10:23.308697+0000","last_active":"2024-03-29T04:10:23.311930+0000","last_peered":"2024-03-29T04:10:23.311930+0000","last_clean":"2024-03-29T04:10:23.311930+0000","last_became_active":"2024-03-29T04:10:23.308430+0000","last_became_peered":"2024-03-29T04:10:23.308430+0000","last_unstale":"2024-03-29T04:10:23.311930+0000","last_undegraded":"2024-03-29T04:10:23.311930+0000","last_fullsized":"2024-03-29T04:10:23.311930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307389+0000","last_change":"2024-03-29T04:10:22.306797+0000","last_active":"2024-03-29T04:10:22.307389+0000","last_peered":"2024-03-29T04:10:22.307389+0000","last_clean":"2024-03-29T04:10:22.307389+0000","last_became_active":"2024-03-29T04:10:22.306504+0000","last_became_peered":"2024-03-29T04:10:22.306504+0000","last_unstale":"2024-03-29T04:10:22.307389+0000","last_undegraded":"2024-03-29T04:10:22.307389+0000","last_fullsized":"2024-03-29T04:10:22.307389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320184+0000","last_change":"2024-03-29T04:10:23.314380+0000","last_active":"2024-03-29T04:10:23.320184+0000","last_peered":"2024-03-29T04:10:23.320184+0000","last_clean":"2024-03-29T04:10:23.320184+0000","last_became_active":"2024-03-29T04:10:23.314129+0000","last_became_peered":"2024-03-29T04:10:23.314129+0000","last_unstale":"2024-03-29T04:10:23.320184+0000","last_undegraded":"2024-03-29T04:10:23.320184+0000","last_fullsized":"2024-03-29T04:10:23.320184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.311633+0000","last_change":"2024-03-29T04:10:22.310915+0000","last_active":"2024-03-29T04:10:22.311633+0000","last_peered":"2024-03-29T04:10:22.311633+0000","last_clean":"2024-03-29T04:10:22.311633+0000","last_became_active":"2024-03-29T04:10:22.310630+0000","last_became_peered":"2024-03-29T04:10:22.310630+0000","last_unstale":"2024-03-29T04:10:22.311633+0000","last_undegraded":"2024-03-29T04:10:22.311633+0000","last_fullsized":"2024-03-29T04:10:22.311633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301511+0000","last_change":"2024-03-29T04:10:22.301511+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301511+0000","last_undegraded":"2024-03-29T04:10:22.301511+0000","last_fullsized":"2024-03-29T04:10:22.301511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319869+0000","last_change":"2024-03-29T04:10:22.304262+0000","last_active":"2024-03-29T04:10:23.319869+0000","last_peered":"2024-03-29T04:10:23.319869+0000","last_clean":"2024-03-29T04:10:23.319869+0000","last_became_active":"2024-03-29T04:10:22.303999+0000","last_became_peered":"2024-03-29T04:10:22.303999+0000","last_unstale":"2024-03-29T04:10:23.319869+0000","last_undegraded":"2024-03-29T04:10:23.319869+0000","last_fullsized":"2024-03-29T04:10:23.319869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292420+0000","last_change":"2024-03-29T04:10:22.292420+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292420+0000","last_undegraded":"2024-03-29T04:10:22.292420+0000","last_fullsized":"2024-03-29T04:10:22.292420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295533+0000","last_change":"2024-03-29T04:10:22.307417+0000","last_active":"2024-03-29T04:10:23.295533+0000","last_peered":"2024-03-29T04:10:23.295533+0000","last_clean":"2024-03-29T04:10:23.295533+0000","last_became_active":"2024-03-29T04:10:22.307123+0000","last_became_peered":"2024-03-29T04:10:22.307123+0000","last_unstale":"2024-03-29T04:10:23.295533+0000","last_undegraded":"2024-03-29T04:10:23.295533+0000","last_fullsized":"2024-03-29T04:10:23.295533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312368+0000","last_change":"2024-03-29T04:10:23.308517+0000","last_active":"2024-03-29T04:10:23.312368+0000","last_peered":"2024-03-29T04:10:23.312368+0000","last_clean":"2024-03-29T04:10:23.312368+0000","last_became_active":"2024-03-29T04:10:23.308180+0000","last_became_peered":"2024-03-29T04:10:23.308180+0000","last_unstale":"2024-03-29T04:10:23.312368+0000","last_undegraded":"2024-03-29T04:10:23.312368+0000","last_fullsized":"2024-03-29T04:10:23.312368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.296007+0000","last_change":"2024-03-29T04:10:22.294851+0000","last_active":"2024-03-29T04:10:22.296007+0000","last_peered":"2024-03-29T04:10:22.296007+0000","last_clean":"2024-03-29T04:10:22.296007+0000","last_became_active":"2024-03-29T04:10:22.294580+0000","last_became_peered":"2024-03-29T04:10:22.294580+0000","last_unstale":"2024-03-29T04:10:22.296007+0000","last_undegraded":"2024-03-29T04:10:22.296007+0000","last_fullsized":"2024-03-29T04:10:22.296007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321112+0000","last_change":"2024-03-29T04:10:23.313712+0000","last_active":"2024-03-29T04:10:23.321112+0000","last_peered":"2024-03-29T04:10:23.321112+0000","last_clean":"2024-03-29T04:10:23.321112+0000","last_became_active":"2024-03-29T04:10:23.313467+0000","last_became_peered":"2024-03-29T04:10:23.313467+0000","last_unstale":"2024-03-29T04:10:23.321112+0000","last_undegraded":"2024-03-29T04:10:23.321112+0000","last_fullsized":"2024-03-29T04:10:23.321112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:39:13.669270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295738+0000","last_change":"2024-03-29T04:10:22.305312+0000","last_active":"2024-03-29T04:10:23.295738+0000","last_peered":"2024-03-29T04:10:23.295738+0000","last_clean":"2024-03-29T04:10:23.295738+0000","last_became_active":"2024-03-29T04:10:22.304984+0000","last_became_peered":"2024-03-29T04:10:22.304984+0000","last_unstale":"2024-03-29T04:10:23.295738+0000","last_undegraded":"2024-03-29T04:10:23.295738+0000","last_fullsized":"2024-03-29T04:10:23.295738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:35.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314641+0000","last_change":"2024-03-29T04:10:23.304934+0000","last_active":"2024-03-29T04:10:23.314641+0000","last_peered":"2024-03-29T04:10:23.314641+0000","last_clean":"2024-03-29T04:10:23.314641+0000","last_became_active":"2024-03-29T04:10:23.304669+0000","last_became_peered":"2024-03-29T04:10:23.304669+0000","last_unstale":"2024-03-29T04:10:23.314641+0000","last_undegraded":"2024-03-29T04:10:23.314641+0000","last_fullsized":"2024-03-29T04:10:23.314641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309574+0000","last_change":"2024-03-29T04:10:22.308050+0000","last_active":"2024-03-29T04:10:22.309574+0000","last_peered":"2024-03-29T04:10:22.309574+0000","last_clean":"2024-03-29T04:10:22.309574+0000","last_became_active":"2024-03-29T04:10:22.307753+0000","last_became_peered":"2024-03-29T04:10:22.307753+0000","last_unstale":"2024-03-29T04:10:22.309574+0000","last_undegraded":"2024-03-29T04:10:22.309574+0000","last_fullsized":"2024-03-29T04:10:22.309574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322352+0000","last_change":"2024-03-29T04:10:23.317220+0000","last_active":"2024-03-29T04:10:23.322352+0000","last_peered":"2024-03-29T04:10:23.322352+0000","last_clean":"2024-03-29T04:10:23.322352+0000","last_became_active":"2024-03-29T04:10:23.316962+0000","last_became_peered":"2024-03-29T04:10:23.316962+0000","last_unstale":"2024-03-29T04:10:23.322352+0000","last_undegraded":"2024-03-29T04:10:23.322352+0000","last_fullsized":"2024-03-29T04:10:23.322352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310684+0000","last_change":"2024-03-29T04:10:22.308995+0000","last_active":"2024-03-29T04:10:22.310684+0000","last_peered":"2024-03-29T04:10:22.310684+0000","last_clean":"2024-03-29T04:10:22.310684+0000","last_became_active":"2024-03-29T04:10:22.308679+0000","last_became_peered":"2024-03-29T04:10:22.308679+0000","last_unstale":"2024-03-29T04:10:22.310684+0000","last_undegraded":"2024-03-29T04:10:22.310684+0000","last_fullsized":"2024-03-29T04:10:22.310684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292102+0000","last_change":"2024-03-29T04:10:22.292102+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292102+0000","last_undegraded":"2024-03-29T04:10:22.292102+0000","last_fullsized":"2024-03-29T04:10:22.292102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323985+0000","last_change":"2024-03-29T04:10:22.306679+0000","last_active":"2024-03-29T04:10:23.323985+0000","last_peered":"2024-03-29T04:10:23.323985+0000","last_clean":"2024-03-29T04:10:23.323985+0000","last_became_active":"2024-03-29T04:10:22.306411+0000","last_became_peered":"2024-03-29T04:10:22.306411+0000","last_unstale":"2024-03-29T04:10:23.323985+0000","last_undegraded":"2024-03-29T04:10:23.323985+0000","last_fullsized":"2024-03-29T04:10:23.323985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:22.259732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322368+0000","last_change":"2024-03-29T04:10:23.316218+0000","last_active":"2024-03-29T04:10:23.322368+0000","last_peered":"2024-03-29T04:10:23.322368+0000","last_clean":"2024-03-29T04:10:23.322368+0000","last_became_active":"2024-03-29T04:10:23.315899+0000","last_became_peered":"2024-03-29T04:10:23.315899+0000","last_unstale":"2024-03-29T04:10:23.322368+0000","last_undegraded":"2024-03-29T04:10:23.322368+0000","last_fullsized":"2024-03-29T04:10:23.322368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322565+0000","last_change":"2024-03-29T04:10:22.310885+0000","last_active":"2024-03-29T04:10:23.322565+0000","last_peered":"2024-03-29T04:10:23.322565+0000","last_clean":"2024-03-29T04:10:23.322565+0000","last_became_active":"2024-03-29T04:10:22.310589+0000","last_became_peered":"2024-03-29T04:10:22.310589+0000","last_unstale":"2024-03-29T04:10:23.322565+0000","last_undegraded":"2024-03-29T04:10:23.322565+0000","last_fullsized":"2024-03-29T04:10:23.322565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:00.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317792+0000","last_change":"2024-03-29T04:10:23.312331+0000","last_active":"2024-03-29T04:10:23.317792+0000","last_peered":"2024-03-29T04:10:23.317792+0000","last_clean":"2024-03-29T04:10:23.317792+0000","last_became_active":"2024-03-29T04:10:23.311961+0000","last_became_peered":"2024-03-29T04:10:23.311961+0000","last_unstale":"2024-03-29T04:10:23.317792+0000","last_undegraded":"2024-03-29T04:10:23.317792+0000","last_fullsized":"2024-03-29T04:10:23.317792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.298233+0000","last_change":"2024-03-29T04:10:22.308119+0000","last_active":"2024-03-29T04:10:23.298233+0000","last_peered":"2024-03-29T04:10:23.298233+0000","last_clean":"2024-03-29T04:10:23.298233+0000","last_became_active":"2024-03-29T04:10:22.307849+0000","last_became_peered":"2024-03-29T04:10:22.307849+0000","last_unstale":"2024-03-29T04:10:23.298233+0000","last_undegraded":"2024-03-29T04:10:23.298233+0000","last_fullsized":"2024-03-29T04:10:23.298233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316478+0000","last_change":"2024-03-29T04:10:23.310446+0000","last_active":"2024-03-29T04:10:23.316478+0000","last_peered":"2024-03-29T04:10:23.316478+0000","last_clean":"2024-03-29T04:10:23.316478+0000","last_became_active":"2024-03-29T04:10:23.310183+0000","last_became_peered":"2024-03-29T04:10:23.310183+0000","last_unstale":"2024-03-29T04:10:23.316478+0000","last_undegraded":"2024-03-29T04:10:23.316478+0000","last_fullsized":"2024-03-29T04:10:23.316478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.291912+0000","last_change":"2024-03-29T04:10:22.297317+0000","last_active":"2024-03-29T04:10:23.291912+0000","last_peered":"2024-03-29T04:10:23.291912+0000","last_clean":"2024-03-29T04:10:23.291912+0000","last_became_active":"2024-03-29T04:10:22.297044+0000","last_became_peered":"2024-03-29T04:10:22.297044+0000","last_unstale":"2024-03-29T04:10:23.291912+0000","last_undegraded":"2024-03-29T04:10:23.291912+0000","last_fullsized":"2024-03-29T04:10:23.291912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:56.164692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317543+0000","last_change":"2024-03-29T04:10:23.312644+0000","last_active":"2024-03-29T04:10:23.317543+0000","last_peered":"2024-03-29T04:10:23.317543+0000","last_clean":"2024-03-29T04:10:23.317543+0000","last_became_active":"2024-03-29T04:10:23.312370+0000","last_became_peered":"2024-03-29T04:10:23.312370+0000","last_unstale":"2024-03-29T04:10:23.317543+0000","last_undegraded":"2024-03-29T04:10:23.317543+0000","last_fullsized":"2024-03-29T04:10:23.317543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320581+0000","last_change":"2024-03-29T04:10:22.304877+0000","last_active":"2024-03-29T04:10:23.320581+0000","last_peered":"2024-03-29T04:10:23.320581+0000","last_clean":"2024-03-29T04:10:23.320581+0000","last_became_active":"2024-03-29T04:10:22.304573+0000","last_became_peered":"2024-03-29T04:10:22.304573+0000","last_unstale":"2024-03-29T04:10:23.320581+0000","last_undegraded":"2024-03-29T04:10:23.320581+0000","last_fullsized":"2024-03-29T04:10:23.320581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.315960+0000","last_change":"2024-03-29T04:10:23.309896+0000","last_active":"2024-03-29T04:10:23.315960+0000","last_peered":"2024-03-29T04:10:23.315960+0000","last_clean":"2024-03-29T04:10:23.315960+0000","last_became_active":"2024-03-29T04:10:23.308057+0000","last_became_peered":"2024-03-29T04:10:23.308057+0000","last_unstale":"2024-03-29T04:10:23.315960+0000","last_undegraded":"2024-03-29T04:10:23.315960+0000","last_fullsized":"2024-03-29T04:10:23.315960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307929+0000","last_change":"2024-03-29T04:10:22.307155+0000","last_active":"2024-03-29T04:10:22.307929+0000","last_peered":"2024-03-29T04:10:22.307929+0000","last_clean":"2024-03-29T04:10:22.307929+0000","last_became_active":"2024-03-29T04:10:22.306817+0000","last_became_peered":"2024-03-29T04:10:22.306817+0000","last_unstale":"2024-03-29T04:10:22.307929+0000","last_undegraded":"2024-03-29T04:10:22.307929+0000","last_fullsized":"2024-03-29T04:10:22.307929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301030+0000","last_change":"2024-03-29T04:10:22.301030+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301030+0000","last_undegraded":"2024-03-29T04:10:22.301030+0000","last_fullsized":"2024-03-29T04:10:22.301030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309562+0000","last_change":"2024-03-29T04:10:22.307690+0000","last_active":"2024-03-29T04:10:22.309562+0000","last_peered":"2024-03-29T04:10:22.309562+0000","last_clean":"2024-03-29T04:10:22.309562+0000","last_became_active":"2024-03-29T04:10:22.307424+0000","last_became_peered":"2024-03-29T04:10:22.307424+0000","last_unstale":"2024-03-29T04:10:22.309562+0000","last_undegraded":"2024-03-29T04:10:22.309562+0000","last_fullsized":"2024-03-29T04:10:22.309562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292383+0000","last_change":"2024-03-29T04:10:22.292383+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292383+0000","last_undegraded":"2024-03-29T04:10:22.292383+0000","last_fullsized":"2024-03-29T04:10:22.292383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318361+0000","last_change":"2024-03-29T04:10:22.303744+0000","last_active":"2024-03-29T04:10:23.318361+0000","last_peered":"2024-03-29T04:10:23.318361+0000","last_clean":"2024-03-29T04:10:23.318361+0000","last_became_active":"2024-03-29T04:10:22.303477+0000","last_became_peered":"2024-03-29T04:10:22.303477+0000","last_unstale":"2024-03-29T04:10:23.318361+0000","last_undegraded":"2024-03-29T04:10:23.318361+0000","last_fullsized":"2024-03-29T04:10:23.318361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:03:07.972943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.310284+0000","last_change":"2024-03-29T04:10:23.306990+0000","last_active":"2024-03-29T04:10:23.310284+0000","last_peered":"2024-03-29T04:10:23.310284+0000","last_clean":"2024-03-29T04:10:23.310284+0000","last_became_active":"2024-03-29T04:10:23.306635+0000","last_became_peered":"2024-03-29T04:10:23.306635+0000","last_unstale":"2024-03-29T04:10:23.310284+0000","last_undegraded":"2024-03-29T04:10:23.310284+0000","last_fullsized":"2024-03-29T04:10:23.310284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:25.613005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297872+0000","last_change":"2024-03-29T04:10:22.300528+0000","last_active":"2024-03-29T04:10:23.297872+0000","last_peered":"2024-03-29T04:10:23.297872+0000","last_clean":"2024-03-29T04:10:23.297872+0000","last_became_active":"2024-03-29T04:10:22.299874+0000","last_became_peered":"2024-03-29T04:10:22.299874+0000","last_unstale":"2024-03-29T04:10:23.297872+0000","last_undegraded":"2024-03-29T04:10:23.297872+0000","last_fullsized":"2024-03-29T04:10:23.297872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316810+0000","last_change":"2024-03-29T04:10:23.311796+0000","last_active":"2024-03-29T04:10:23.316810+0000","last_peered":"2024-03-29T04:10:23.316810+0000","last_clean":"2024-03-29T04:10:23.316810+0000","last_became_active":"2024-03-29T04:10:23.311351+0000","last_became_peered":"2024-03-29T04:10:23.311351+0000","last_unstale":"2024-03-29T04:10:23.316810+0000","last_undegraded":"2024-03-29T04:10:23.316810+0000","last_fullsized":"2024-03-29T04:10:23.316810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309583+0000","last_change":"2024-03-29T04:10:22.308134+0000","last_active":"2024-03-29T04:10:23.309583+0000","last_peered":"2024-03-29T04:10:23.309583+0000","last_clean":"2024-03-29T04:10:23.309583+0000","last_became_active":"2024-03-29T04:10:22.307860+0000","last_became_peered":"2024-03-29T04:10:22.307860+0000","last_unstale":"2024-03-29T04:10:23.309583+0000","last_undegraded":"2024-03-29T04:10:23.309583+0000","last_fullsized":"2024-03-29T04:10:23.309583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:20:17.587318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311340+0000","last_change":"2024-03-29T04:10:23.309927+0000","last_active":"2024-03-29T04:10:23.311340+0000","last_peered":"2024-03-29T04:10:23.311340+0000","last_clean":"2024-03-29T04:10:23.311340+0000","last_became_active":"2024-03-29T04:10:23.309623+0000","last_became_peered":"2024-03-29T04:10:23.309623+0000","last_unstale":"2024-03-29T04:10:23.311340+0000","last_undegraded":"2024-03-29T04:10:23.311340+0000","last_fullsized":"2024-03-29T04:10:23.311340+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.301262+0000","last_change":"2024-03-29T04:10:22.297913+0000","last_active":"2024-03-29T04:10:23.301262+0000","last_peered":"2024-03-29T04:10:23.301262+0000","last_clean":"2024-03-29T04:10:23.301262+0000","last_became_active":"2024-03-29T04:10:22.296015+0000","last_became_peered":"2024-03-29T04:10:22.296015+0000","last_unstale":"2024-03-29T04:10:23.301262+0000","last_undegraded":"2024-03-29T04:10:23.301262+0000","last_fullsized":"2024-03-29T04:10:23.301262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:13.154992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300580+0000","last_change":"2024-03-29T04:10:22.300580+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300580+0000","last_undegraded":"2024-03-29T04:10:22.300580+0000","last_fullsized":"2024-03-29T04:10:22.300580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292907+0000","last_change":"2024-03-29T04:10:22.305287+0000","last_active":"2024-03-29T04:10:23.292907+0000","last_peered":"2024-03-29T04:10:23.292907+0000","last_clean":"2024-03-29T04:10:23.292907+0000","last_became_active":"2024-03-29T04:10:22.304999+0000","last_became_peered":"2024-03-29T04:10:22.304999+0000","last_unstale":"2024-03-29T04:10:23.292907+0000","last_undegraded":"2024-03-29T04:10:23.292907+0000","last_fullsized":"2024-03-29T04:10:23.292907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312347+0000","last_change":"2024-03-29T04:10:23.307638+0000","last_active":"2024-03-29T04:10:23.312347+0000","last_peered":"2024-03-29T04:10:23.312347+0000","last_clean":"2024-03-29T04:10:23.312347+0000","last_became_active":"2024-03-29T04:10:23.307381+0000","last_became_peered":"2024-03-29T04:10:23.307381+0000","last_unstale":"2024-03-29T04:10:23.312347+0000","last_undegraded":"2024-03-29T04:10:23.312347+0000","last_fullsized":"2024-03-29T04:10:23.312347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299841+0000","last_change":"2024-03-29T04:10:22.307254+0000","last_active":"2024-03-29T04:10:23.299841+0000","last_peered":"2024-03-29T04:10:23.299841+0000","last_clean":"2024-03-29T04:10:23.299841+0000","last_became_active":"2024-03-29T04:10:22.306976+0000","last_became_peered":"2024-03-29T04:10:22.306976+0000","last_unstale":"2024-03-29T04:10:23.299841+0000","last_undegraded":"2024-03-29T04:10:23.299841+0000","last_fullsized":"2024-03-29T04:10:23.299841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:54:11.970574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320128+0000","last_change":"2024-03-29T04:10:23.317609+0000","last_active":"2024-03-29T04:10:23.320128+0000","last_peered":"2024-03-29T04:10:23.320128+0000","last_clean":"2024-03-29T04:10:23.320128+0000","last_became_active":"2024-03-29T04:10:23.317364+0000","last_became_peered":"2024-03-29T04:10:23.317364+0000","last_unstale":"2024-03-29T04:10:23.320128+0000","last_undegraded":"2024-03-29T04:10:23.320128+0000","last_fullsized":"2024-03-29T04:10:23.320128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310663+0000","last_change":"2024-03-29T04:10:22.308920+0000","last_active":"2024-03-29T04:10:22.310663+0000","last_peered":"2024-03-29T04:10:22.310663+0000","last_clean":"2024-03-29T04:10:22.310663+0000","last_became_active":"2024-03-29T04:10:22.308648+0000","last_became_peered":"2024-03-29T04:10:22.308648+0000","last_unstale":"2024-03-29T04:10:22.310663+0000","last_undegraded":"2024-03-29T04:10:22.310663+0000","last_fullsized":"2024-03-29T04:10:22.310663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314664+0000","last_change":"2024-03-29T04:10:23.305533+0000","last_active":"2024-03-29T04:10:23.314664+0000","last_peered":"2024-03-29T04:10:23.314664+0000","last_clean":"2024-03-29T04:10:23.314664+0000","last_became_active":"2024-03-29T04:10:23.305295+0000","last_became_peered":"2024-03-29T04:10:23.305295+0000","last_unstale":"2024-03-29T04:10:23.314664+0000","last_undegraded":"2024-03-29T04:10:23.314664+0000","last_fullsized":"2024-03-29T04:10:23.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304196+0000","last_change":"2024-03-29T04:10:22.309267+0000","last_active":"2024-03-29T04:10:23.304196+0000","last_peered":"2024-03-29T04:10:23.304196+0000","last_clean":"2024-03-29T04:10:23.304196+0000","last_became_active":"2024-03-29T04:10:22.308996+0000","last_became_peered":"2024-03-29T04:10:22.308996+0000","last_unstale":"2024-03-29T04:10:23.304196+0000","last_undegraded":"2024-03-29T04:10:23.304196+0000","last_fullsized":"2024-03-29T04:10:23.304196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:27.354497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317732+0000","last_change":"2024-03-29T04:10:23.313551+0000","last_active":"2024-03-29T04:10:23.317732+0000","last_peered":"2024-03-29T04:10:23.317732+0000","last_clean":"2024-03-29T04:10:23.317732+0000","last_became_active":"2024-03-29T04:10:23.313289+0000","last_became_peered":"2024-03-29T04:10:23.313289+0000","last_unstale":"2024-03-29T04:10:23.317732+0000","last_undegraded":"2024-03-29T04:10:23.317732+0000","last_fullsized":"2024-03-29T04:10:23.317732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299143+0000","last_change":"2024-03-29T04:10:22.308648+0000","last_active":"2024-03-29T04:10:23.299143+0000","last_peered":"2024-03-29T04:10:23.299143+0000","last_clean":"2024-03-29T04:10:23.299143+0000","last_became_active":"2024-03-29T04:10:22.308334+0000","last_became_peered":"2024-03-29T04:10:22.308334+0000","last_unstale":"2024-03-29T04:10:23.299143+0000","last_undegraded":"2024-03-29T04:10:23.299143+0000","last_fullsized":"2024-03-29T04:10:23.299143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322380+0000","last_change":"2024-03-29T04:10:23.319502+0000","last_active":"2024-03-29T04:10:23.322380+0000","last_peered":"2024-03-29T04:10:23.322380+0000","last_clean":"2024-03-29T04:10:23.322380+0000","last_became_active":"2024-03-29T04:10:23.319209+0000","last_became_peered":"2024-03-29T04:10:23.319209+0000","last_unstale":"2024-03-29T04:10:23.322380+0000","last_undegraded":"2024-03-29T04:10:23.322380+0000","last_fullsized":"2024-03-29T04:10:23.322380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.306085+0000","last_change":"2024-03-29T04:10:22.307995+0000","last_active":"2024-03-29T04:10:23.306085+0000","last_peered":"2024-03-29T04:10:23.306085+0000","last_clean":"2024-03-29T04:10:23.306085+0000","last_became_active":"2024-03-29T04:10:22.307723+0000","last_became_peered":"2024-03-29T04:10:22.307723+0000","last_unstale":"2024-03-29T04:10:23.306085+0000","last_undegraded":"2024-03-29T04:10:23.306085+0000","last_fullsized":"2024-03-29T04:10:23.306085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316387+0000","last_change":"2024-03-29T04:10:23.308586+0000","last_active":"2024-03-29T04:10:23.316387+0000","last_peered":"2024-03-29T04:10:23.316387+0000","last_clean":"2024-03-29T04:10:23.316387+0000","last_became_active":"2024-03-29T04:10:23.308278+0000","last_became_peered":"2024-03-29T04:10:23.308278+0000","last_unstale":"2024-03-29T04:10:23.316387+0000","last_undegraded":"2024-03-29T04:10:23.316387+0000","last_fullsized":"2024-03-29T04:10:23.316387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295435+0000","last_change":"2024-03-29T04:10:22.309876+0000","last_active":"2024-03-29T04:10:23.295435+0000","last_peered":"2024-03-29T04:10:23.295435+0000","last_clean":"2024-03-29T04:10:23.295435+0000","last_became_active":"2024-03-29T04:10:22.309473+0000","last_became_peered":"2024-03-29T04:10:22.309473+0000","last_unstale":"2024-03-29T04:10:23.295435+0000","last_undegraded":"2024-03-29T04:10:23.295435+0000","last_fullsized":"2024-03-29T04:10:23.295435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:20.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292868+0000","last_change":"2024-03-29T04:10:22.292868+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292868+0000","last_undegraded":"2024-03-29T04:10:22.292868+0000","last_fullsized":"2024-03-29T04:10:22.292868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.303802+0000","last_change":"2024-03-29T04:10:22.302187+0000","last_active":"2024-03-29T04:10:22.303802+0000","last_peered":"2024-03-29T04:10:22.303802+0000","last_clean":"2024-03-29T04:10:22.303802+0000","last_became_active":"2024-03-29T04:10:22.301830+0000","last_became_peered":"2024-03-29T04:10:22.301830+0000","last_unstale":"2024-03-29T04:10:22.303802+0000","last_undegraded":"2024-03-29T04:10:22.303802+0000","last_fullsized":"2024-03-29T04:10:22.303802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314171+0000","last_change":"2024-03-29T04:10:23.303127+0000","last_active":"2024-03-29T04:10:23.314171+0000","last_peered":"2024-03-29T04:10:23.314171+0000","last_clean":"2024-03-29T04:10:23.314171+0000","last_became_active":"2024-03-29T04:10:23.302882+0000","last_became_peered":"2024-03-29T04:10:23.302882+0000","last_unstale":"2024-03-29T04:10:23.314171+0000","last_undegraded":"2024-03-29T04:10:23.314171+0000","last_fullsized":"2024-03-29T04:10:23.314171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297157+0000","last_change":"2024-03-29T04:10:22.304632+0000","last_active":"2024-03-29T04:10:23.297157+0000","last_peered":"2024-03-29T04:10:23.297157+0000","last_clean":"2024-03-29T04:10:23.297157+0000","last_became_active":"2024-03-29T04:10:22.304363+0000","last_became_peered":"2024-03-29T04:10:22.304363+0000","last_unstale":"2024-03-29T04:10:23.297157+0000","last_undegraded":"2024-03-29T04:10:23.297157+0000","last_fullsized":"2024-03-29T04:10:23.297157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:35.091291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301182+0000","last_change":"2024-03-29T04:10:22.301182+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301182+0000","last_undegraded":"2024-03-29T04:10:22.301182+0000","last_fullsized":"2024-03-29T04:10:22.301182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.293779+0000","last_change":"2024-03-29T04:10:22.310649+0000","last_active":"2024-03-29T04:10:23.293779+0000","last_peered":"2024-03-29T04:10:23.293779+0000","last_clean":"2024-03-29T04:10:23.293779+0000","last_became_active":"2024-03-29T04:10:22.310382+0000","last_became_peered":"2024-03-29T04:10:22.310382+0000","last_unstale":"2024-03-29T04:10:23.293779+0000","last_undegraded":"2024-03-29T04:10:23.293779+0000","last_fullsized":"2024-03-29T04:10:23.293779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292459+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:23.292459+0000","last_peered":"2024-03-29T04:10:23.292459+0000","last_clean":"2024-03-29T04:10:23.292459+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:23.292459+0000","last_undegraded":"2024-03-29T04:10:23.292459+0000","last_fullsized":"2024-03-29T04:10:23.292459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":192127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:26.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.625+0000 7efc2237c640 1 Processor -- start 2024-03-29T04:10:26.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.625+0000 7efc2237c640 1 -- start start 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc2237c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1c0cccf0 0x7efc1c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc2237c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc1c0d0f10 0x7efc1c0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc2237c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc1c0cfe70 con 0x7efc1c152dc0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc2237c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7efc1c153930 con 0x7efc1c0cccf0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc2237c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc1c0d0150 con 0x7efc1c0d0f10 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1c0cccf0 0x7efc1c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc1c0cccf0 0x7efc1c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55128/0 (socket says 172.21.15.181:55128) 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 -- 172.21.15.181:0/2076153286 learned_addr learned my addr 172.21.15.181:0/2076153286 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:26.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1b7fe640 1 --2- 172.21.15.181:0/2076153286 >> v2:172.21.15.187:3300/0 conn(0x7efc1c0d0f10 0x7efc1c0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 -- 172.21.15.181:0/2076153286 >> v2:172.21.15.187:3300/0 conn(0x7efc1c0d0f10 msgr2=0x7efc1c0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 --2- 172.21.15.181:0/2076153286 >> v2:172.21.15.187:3300/0 conn(0x7efc1c0d0f10 0x7efc1c0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 -- 172.21.15.181:0/2076153286 >> v1:172.21.15.187:6789/0 conn(0x7efc1c152dc0 legacy=0x7efc1c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.626+0000 7efc1bfff640 1 -- 172.21.15.181:0/2076153286 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c0d0430 con 0x7efc1c0cccf0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.627+0000 7efc1bfff640 1 --2- 172.21.15.181:0/2076153286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0cccf0 0x7efc1c0b3d20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc1c068200 tx=0x7efc1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b907c69e76e5504d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.627+0000 7efc1affd640 1 -- 172.21.15.181:0/2076153286 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10034070 con 0x7efc1c0cccf0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.627+0000 7efc1affd640 1 -- 172.21.15.181:0/2076153286 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc10030860 con 0x7efc1c0cccf0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.627+0000 7efc1affd640 1 -- 172.21.15.181:0/2076153286 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10030c30 con 0x7efc1c0cccf0 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 -- 172.21.15.181:0/2076153286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0cccf0 msgr2=0x7efc1c0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 --2- 172.21.15.181:0/2076153286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0cccf0 0x7efc1c0b3d20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc1c068200 tx=0x7efc1002b9a0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 -- 172.21.15.181:0/2076153286 shutdown_connections 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 --2- 172.21.15.181:0/2076153286 >> v2:172.21.15.187:3300/0 conn(0x7efc1c0d0f10 0x7efc1c0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.608 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 --2- 172.21.15.181:0/2076153286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0cccf0 0x7efc1c0b3d20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 -- 172.21.15.181:0/2076153286 >> 172.21.15.181:0/2076153286 conn(0x7efc1c0672c0 msgr2=0x7efc1c0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 -- 172.21.15.181:0/2076153286 shutdown_connections 2024-03-29T04:10:26.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.628+0000 7efc2237c640 1 -- 172.21.15.181:0/2076153286 wait complete. 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.629+0000 7efc2237c640 1 Processor -- start 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.629+0000 7efc2237c640 1 -- start start 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc1c0cccf0 0x7efc1c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c152dc0 0x7efc1c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c0cd0d0 con 0x7efc1c0d0f10 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c0d09f0 con 0x7efc1c152dc0 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc2237c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c0cd250 con 0x7efc1c0cccf0 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55136/0 (socket says 172.21.15.181:55136) 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc1b7fe640 1 -- 172.21.15.181:0/655132232 learned_addr learned my addr 172.21.15.181:0/655132232 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc1bfff640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc1c0cccf0 0x7efc1c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.630+0000 7efc20919640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c152dc0 0x7efc1c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 -- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc1c0cccf0 msgr2=0x7efc1c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc1c0cccf0 0x7efc1c0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 -- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c152dc0 msgr2=0x7efc1c14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c152dc0 0x7efc1c14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 -- 172.21.15.181:0/655132232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c1f2e50 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc1b7fe640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00d5c0 tx=0x7efc0c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0125f0 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0c019070 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0129d0 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c1f3130 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.631+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c0b12e0 con 0x7efc1c0d0f10 2024-03-29T04:10:26.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.632+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1c075a10 con 0x7efc1c0d0f10 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.632+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc0c012b70 con 0x7efc1c0d0f10 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.633+0000 7efc18ff9640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 0x7efc0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.633+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7efc0c08bb00 con 0x7efc1c0d0f10 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.633+0000 7efc1bfff640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 0x7efc0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.633+0000 7efc1bfff640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 0x7efc0806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc10036000 tx=0x7efc1003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.641+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc0c059940 con 0x7efc1c0d0f10 2024-03-29T04:10:26.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.819+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc1c1f2bf0 con 0x7efc0806c530 2024-03-29T04:10:26.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.825+0000 7efc18ff9640 1 -- 172.21.15.181:0/655132232 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250415 (secure 0 0 0) 0x7efc10030880 con 0x7efc0806c530 2024-03-29T04:10:26.810 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:26.812 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:10:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 msgr2=0x7efc0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.815 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 0x7efc0806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc10036000 tx=0x7efc1003b040 comp rx=0 tx=0).stop 2024-03-29T04:10:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 msgr2=0x7efc1c0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00d5c0 tx=0x7efc0c00da80 comp rx=0 tx=0).stop 2024-03-29T04:10:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 shutdown_connections 2024-03-29T04:10:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc1c152dc0 0x7efc1c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.816 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc1c0d0f10 0x7efc1c0b0da0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7efc0806c530 0x7efc0806e9f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 --2- 172.21.15.181:0/655132232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc1c0cccf0 0x7efc1c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.828+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 >> 172.21.15.181:0/655132232 conn(0x7efc1c0672c0 msgr2=0x7efc1c0b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.829+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 shutdown_connections 2024-03-29T04:10:26.817 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:26.829+0000 7efc2237c640 1 -- 172.21.15.181:0/655132232 wait complete. 2024-03-29T04:10:26.819 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-03-29T04:10:25.587035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220684,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754036,"statfs":{"total":773094113280,"available":772868132864,"internally_reserved":0,"allocated":4616192,"data_stored":2649928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.305977"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318073+0000","last_change":"2024-03-29T04:10:23.313049+0000","last_active":"2024-03-29T04:10:23.318073+0000","last_peered":"2024-03-29T04:10:23.318073+0000","last_clean":"2024-03-29T04:10:23.318073+0000","last_became_active":"2024-03-29T04:10:23.312619+0000","last_became_peered":"2024-03-29T04:10:23.312619+0000","last_unstale":"2024-03-29T04:10:23.318073+0000","last_undegraded":"2024-03-29T04:10:23.318073+0000","last_fullsized":"2024-03-29T04:10:23.318073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309796+0000","last_change":"2024-03-29T04:10:23.304958+0000","last_active":"2024-03-29T04:10:23.309796+0000","last_peered":"2024-03-29T04:10:23.309796+0000","last_clean":"2024-03-29T04:10:23.309796+0000","last_became_active":"2024-03-29T04:10:23.304714+0000","last_became_peered":"2024-03-29T04:10:23.304714+0000","last_unstale":"2024-03-29T04:10:23.309796+0000","last_undegraded":"2024-03-29T04:10:23.309796+0000","last_fullsized":"2024-03-29T04:10:23.309796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322299+0000","last_change":"2024-03-29T04:10:23.315527+0000","last_active":"2024-03-29T04:10:23.322299+0000","last_peered":"2024-03-29T04:10:23.322299+0000","last_clean":"2024-03-29T04:10:23.322299+0000","last_became_active":"2024-03-29T04:10:23.315242+0000","last_became_peered":"2024-03-29T04:10:23.315242+0000","last_unstale":"2024-03-29T04:10:23.322299+0000","last_undegraded":"2024-03-29T04:10:23.322299+0000","last_fullsized":"2024-03-29T04:10:23.322299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321235+0000","last_change":"2024-03-29T04:10:23.315865+0000","last_active":"2024-03-29T04:10:23.321235+0000","last_peered":"2024-03-29T04:10:23.321235+0000","last_clean":"2024-03-29T04:10:23.321235+0000","last_became_active":"2024-03-29T04:10:23.315580+0000","last_became_peered":"2024-03-29T04:10:23.315580+0000","last_unstale":"2024-03-29T04:10:23.321235+0000","last_undegraded":"2024-03-29T04:10:23.321235+0000","last_fullsized":"2024-03-29T04:10:23.321235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:33:01.952778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321146+0000","last_change":"2024-03-29T04:10:23.315150+0000","last_active":"2024-03-29T04:10:23.321146+0000","last_peered":"2024-03-29T04:10:23.321146+0000","last_clean":"2024-03-29T04:10:23.321146+0000","last_became_active":"2024-03-29T04:10:23.314879+0000","last_became_peered":"2024-03-29T04:10:23.314879+0000","last_unstale":"2024-03-29T04:10:23.321146+0000","last_undegraded":"2024-03-29T04:10:23.321146+0000","last_fullsized":"2024-03-29T04:10:23.321146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.011645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316771+0000","last_change":"2024-03-29T04:10:23.311220+0000","last_active":"2024-03-29T04:10:23.316771+0000","last_peered":"2024-03-29T04:10:23.316771+0000","last_clean":"2024-03-29T04:10:23.316771+0000","last_became_active":"2024-03-29T04:10:23.310727+0000","last_became_peered":"2024-03-29T04:10:23.310727+0000","last_unstale":"2024-03-29T04:10:23.316771+0000","last_undegraded":"2024-03-29T04:10:23.316771+0000","last_fullsized":"2024-03-29T04:10:23.316771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.302395+0000","last_change":"2024-03-29T04:10:22.302395+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.302395+0000","last_undegraded":"2024-03-29T04:10:22.302395+0000","last_fullsized":"2024-03-29T04:10:22.302395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317690+0000","last_change":"2024-03-29T04:10:23.314124+0000","last_active":"2024-03-29T04:10:23.317690+0000","last_peered":"2024-03-29T04:10:23.317690+0000","last_clean":"2024-03-29T04:10:23.317690+0000","last_became_active":"2024-03-29T04:10:23.313434+0000","last_became_peered":"2024-03-29T04:10:23.313434+0000","last_unstale":"2024-03-29T04:10:23.317690+0000","last_undegraded":"2024-03-29T04:10:23.317690+0000","last_fullsized":"2024-03-29T04:10:23.317690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322272+0000","last_change":"2024-03-29T04:10:23.315461+0000","last_active":"2024-03-29T04:10:23.322272+0000","last_peered":"2024-03-29T04:10:23.322272+0000","last_clean":"2024-03-29T04:10:23.322272+0000","last_became_active":"2024-03-29T04:10:23.315163+0000","last_became_peered":"2024-03-29T04:10:23.315163+0000","last_unstale":"2024-03-29T04:10:23.322272+0000","last_undegraded":"2024-03-29T04:10:23.322272+0000","last_fullsized":"2024-03-29T04:10:23.322272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.303882+0000","last_change":"2024-03-29T04:10:22.303882+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.303882+0000","last_undegraded":"2024-03-29T04:10:22.303882+0000","last_fullsized":"2024-03-29T04:10:22.303882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301429+0000","last_change":"2024-03-29T04:10:22.301429+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301429+0000","last_undegraded":"2024-03-29T04:10:22.301429+0000","last_fullsized":"2024-03-29T04:10:22.301429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311095+0000","last_change":"2024-03-29T04:10:23.309450+0000","last_active":"2024-03-29T04:10:23.311095+0000","last_peered":"2024-03-29T04:10:23.311095+0000","last_clean":"2024-03-29T04:10:23.311095+0000","last_became_active":"2024-03-29T04:10:23.309186+0000","last_became_peered":"2024-03-29T04:10:23.309186+0000","last_unstale":"2024-03-29T04:10:23.311095+0000","last_undegraded":"2024-03-29T04:10:23.311095+0000","last_fullsized":"2024-03-29T04:10:23.311095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.135609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300951+0000","last_change":"2024-03-29T04:10:22.300951+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300951+0000","last_undegraded":"2024-03-29T04:10:22.300951+0000","last_fullsized":"2024-03-29T04:10:22.300951+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.293775+0000","last_change":"2024-03-29T04:10:22.293775+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.293775+0000","last_undegraded":"2024-03-29T04:10:22.293775+0000","last_fullsized":"2024-03-29T04:10:22.293775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301942+0000","last_change":"2024-03-29T04:10:22.301942+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301942+0000","last_undegraded":"2024-03-29T04:10:22.301942+0000","last_fullsized":"2024-03-29T04:10:22.301942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316831+0000","last_change":"2024-03-29T04:10:23.311958+0000","last_active":"2024-03-29T04:10:23.316831+0000","last_peered":"2024-03-29T04:10:23.316831+0000","last_clean":"2024-03-29T04:10:23.316831+0000","last_became_active":"2024-03-29T04:10:23.311700+0000","last_became_peered":"2024-03-29T04:10:23.311700+0000","last_unstale":"2024-03-29T04:10:23.316831+0000","last_undegraded":"2024-03-29T04:10:23.316831+0000","last_fullsized":"2024-03-29T04:10:23.316831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316930+0000","last_change":"2024-03-29T04:10:23.313386+0000","last_active":"2024-03-29T04:10:23.316930+0000","last_peered":"2024-03-29T04:10:23.316930+0000","last_clean":"2024-03-29T04:10:23.316930+0000","last_became_active":"2024-03-29T04:10:23.313083+0000","last_became_peered":"2024-03-29T04:10:23.313083+0000","last_unstale":"2024-03-29T04:10:23.316930+0000","last_undegraded":"2024-03-29T04:10:23.316930+0000","last_fullsized":"2024-03-29T04:10:23.316930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304763+0000","last_change":"2024-03-29T04:10:23.300122+0000","last_active":"2024-03-29T04:10:23.304763+0000","last_peered":"2024-03-29T04:10:23.304763+0000","last_clean":"2024-03-29T04:10:23.304763+0000","last_became_active":"2024-03-29T04:10:23.299864+0000","last_became_peered":"2024-03-29T04:10:23.299864+0000","last_unstale":"2024-03-29T04:10:23.304763+0000","last_undegraded":"2024-03-29T04:10:23.304763+0000","last_fullsized":"2024-03-29T04:10:23.304763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:29.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321486+0000","last_change":"2024-03-29T04:10:23.316162+0000","last_active":"2024-03-29T04:10:23.321486+0000","last_peered":"2024-03-29T04:10:23.321486+0000","last_clean":"2024-03-29T04:10:23.321486+0000","last_became_active":"2024-03-29T04:10:23.315894+0000","last_became_peered":"2024-03-29T04:10:23.315894+0000","last_unstale":"2024-03-29T04:10:23.321486+0000","last_undegraded":"2024-03-29T04:10:23.321486+0000","last_fullsized":"2024-03-29T04:10:23.321486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:37.304758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.305943+0000","last_change":"2024-03-29T04:10:23.297888+0000","last_active":"2024-03-29T04:10:23.305943+0000","last_peered":"2024-03-29T04:10:23.305943+0000","last_clean":"2024-03-29T04:10:23.305943+0000","last_became_active":"2024-03-29T04:10:23.297611+0000","last_became_peered":"2024-03-29T04:10:23.297611+0000","last_unstale":"2024-03-29T04:10:23.305943+0000","last_undegraded":"2024-03-29T04:10:23.305943+0000","last_fullsized":"2024-03-29T04:10:23.305943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:51:35.316362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300543+0000","last_change":"2024-03-29T04:10:22.300543+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300543+0000","last_undegraded":"2024-03-29T04:10:22.300543+0000","last_fullsized":"2024-03-29T04:10:22.300543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291376+0000","last_change":"2024-03-29T04:10:22.291376+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291376+0000","last_undegraded":"2024-03-29T04:10:22.291376+0000","last_fullsized":"2024-03-29T04:10:22.291376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317436+0000","last_change":"2024-03-29T04:10:23.315012+0000","last_active":"2024-03-29T04:10:23.317436+0000","last_peered":"2024-03-29T04:10:23.317436+0000","last_clean":"2024-03-29T04:10:23.317436+0000","last_became_active":"2024-03-29T04:10:23.312553+0000","last_became_peered":"2024-03-29T04:10:23.312553+0000","last_unstale":"2024-03-29T04:10:23.317436+0000","last_undegraded":"2024-03-29T04:10:23.317436+0000","last_fullsized":"2024-03-29T04:10:23.317436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:36.093598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322166+0000","last_change":"2024-03-29T04:10:23.318761+0000","last_active":"2024-03-29T04:10:23.322166+0000","last_peered":"2024-03-29T04:10:23.322166+0000","last_clean":"2024-03-29T04:10:23.322166+0000","last_became_active":"2024-03-29T04:10:23.318414+0000","last_became_peered":"2024-03-29T04:10:23.318414+0000","last_unstale":"2024-03-29T04:10:23.322166+0000","last_undegraded":"2024-03-29T04:10:23.322166+0000","last_fullsized":"2024-03-29T04:10:23.322166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:40.265277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319387+0000","last_change":"2024-03-29T04:10:23.312068+0000","last_active":"2024-03-29T04:10:23.319387+0000","last_peered":"2024-03-29T04:10:23.319387+0000","last_clean":"2024-03-29T04:10:23.319387+0000","last_became_active":"2024-03-29T04:10:23.311776+0000","last_became_peered":"2024-03-29T04:10:23.311776+0000","last_unstale":"2024-03-29T04:10:23.319387+0000","last_undegraded":"2024-03-29T04:10:23.319387+0000","last_fullsized":"2024-03-29T04:10:23.319387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:59.921229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320160+0000","last_change":"2024-03-29T04:10:23.318246+0000","last_active":"2024-03-29T04:10:23.320160+0000","last_peered":"2024-03-29T04:10:23.320160+0000","last_clean":"2024-03-29T04:10:23.320160+0000","last_became_active":"2024-03-29T04:10:23.317989+0000","last_became_peered":"2024-03-29T04:10:23.317989+0000","last_unstale":"2024-03-29T04:10:23.320160+0000","last_undegraded":"2024-03-29T04:10:23.320160+0000","last_fullsized":"2024-03-29T04:10:23.320160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309790+0000","last_change":"2024-03-29T04:10:23.308341+0000","last_active":"2024-03-29T04:10:23.309790+0000","last_peered":"2024-03-29T04:10:23.309790+0000","last_clean":"2024-03-29T04:10:23.309790+0000","last_became_active":"2024-03-29T04:10:23.307789+0000","last_became_peered":"2024-03-29T04:10:23.307789+0000","last_unstale":"2024-03-29T04:10:23.309790+0000","last_undegraded":"2024-03-29T04:10:23.309790+0000","last_fullsized":"2024-03-29T04:10:23.309790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:08.599452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.299555+0000","last_change":"2024-03-29T04:10:22.299555+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.299555+0000","last_undegraded":"2024-03-29T04:10:22.299555+0000","last_fullsized":"2024-03-29T04:10:22.299555+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308971+0000","last_change":"2024-03-29T04:10:23.307876+0000","last_active":"2024-03-29T04:10:23.308971+0000","last_peered":"2024-03-29T04:10:23.308971+0000","last_clean":"2024-03-29T04:10:23.308971+0000","last_became_active":"2024-03-29T04:10:23.307565+0000","last_became_peered":"2024-03-29T04:10:23.307565+0000","last_unstale":"2024-03-29T04:10:23.308971+0000","last_undegraded":"2024-03-29T04:10:23.308971+0000","last_fullsized":"2024-03-29T04:10:23.308971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:11.709686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308760+0000","last_change":"2024-03-29T04:10:23.306970+0000","last_active":"2024-03-29T04:10:23.308760+0000","last_peered":"2024-03-29T04:10:23.308760+0000","last_clean":"2024-03-29T04:10:23.308760+0000","last_became_active":"2024-03-29T04:10:23.306708+0000","last_became_peered":"2024-03-29T04:10:23.306708+0000","last_unstale":"2024-03-29T04:10:23.308760+0000","last_undegraded":"2024-03-29T04:10:23.308760+0000","last_fullsized":"2024-03-29T04:10:23.308760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:15.611882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317417+0000","last_change":"2024-03-29T04:10:23.314775+0000","last_active":"2024-03-29T04:10:23.317417+0000","last_peered":"2024-03-29T04:10:23.317417+0000","last_clean":"2024-03-29T04:10:23.317417+0000","last_became_active":"2024-03-29T04:10:23.312160+0000","last_became_peered":"2024-03-29T04:10:23.312160+0000","last_unstale":"2024-03-29T04:10:23.317417+0000","last_undegraded":"2024-03-29T04:10:23.317417+0000","last_fullsized":"2024-03-29T04:10:23.317417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.313388+0000","last_change":"2024-03-29T04:10:23.304939+0000","last_active":"2024-03-29T04:10:23.313388+0000","last_peered":"2024-03-29T04:10:23.313388+0000","last_clean":"2024-03-29T04:10:23.313388+0000","last_became_active":"2024-03-29T04:10:23.304682+0000","last_became_peered":"2024-03-29T04:10:23.304682+0000","last_unstale":"2024-03-29T04:10:23.313388+0000","last_undegraded":"2024-03-29T04:10:23.313388+0000","last_fullsized":"2024-03-29T04:10:23.313388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323623+0000","last_change":"2024-03-29T04:10:23.318116+0000","last_active":"2024-03-29T04:10:23.323623+0000","last_peered":"2024-03-29T04:10:23.323623+0000","last_clean":"2024-03-29T04:10:23.323623+0000","last_became_active":"2024-03-29T04:10:23.317859+0000","last_became_peered":"2024-03-29T04:10:23.317859+0000","last_unstale":"2024-03-29T04:10:23.323623+0000","last_undegraded":"2024-03-29T04:10:23.323623+0000","last_fullsized":"2024-03-29T04:10:23.323623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319317+0000","last_change":"2024-03-29T04:10:22.297429+0000","last_active":"2024-03-29T04:10:23.319317+0000","last_peered":"2024-03-29T04:10:23.319317+0000","last_clean":"2024-03-29T04:10:23.319317+0000","last_became_active":"2024-03-29T04:10:22.297077+0000","last_became_peered":"2024-03-29T04:10:22.297077+0000","last_unstale":"2024-03-29T04:10:23.319317+0000","last_undegraded":"2024-03-29T04:10:23.319317+0000","last_fullsized":"2024-03-29T04:10:23.319317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:10.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320194+0000","last_change":"2024-03-29T04:10:23.316949+0000","last_active":"2024-03-29T04:10:23.320194+0000","last_peered":"2024-03-29T04:10:23.320194+0000","last_clean":"2024-03-29T04:10:23.320194+0000","last_became_active":"2024-03-29T04:10:23.316665+0000","last_became_peered":"2024-03-29T04:10:23.316665+0000","last_unstale":"2024-03-29T04:10:23.320194+0000","last_undegraded":"2024-03-29T04:10:23.320194+0000","last_fullsized":"2024-03-29T04:10:23.320194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316490+0000","last_change":"2024-03-29T04:10:22.307974+0000","last_active":"2024-03-29T04:10:23.316490+0000","last_peered":"2024-03-29T04:10:23.316490+0000","last_clean":"2024-03-29T04:10:23.316490+0000","last_became_active":"2024-03-29T04:10:22.306547+0000","last_became_peered":"2024-03-29T04:10:22.306547+0000","last_unstale":"2024-03-29T04:10:23.316490+0000","last_undegraded":"2024-03-29T04:10:23.316490+0000","last_fullsized":"2024-03-29T04:10:23.316490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291935+0000","last_change":"2024-03-29T04:10:22.291935+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291935+0000","last_undegraded":"2024-03-29T04:10:22.291935+0000","last_fullsized":"2024-03-29T04:10:22.291935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319454+0000","last_change":"2024-03-29T04:10:22.301842+0000","last_active":"2024-03-29T04:10:23.319454+0000","last_peered":"2024-03-29T04:10:23.319454+0000","last_clean":"2024-03-29T04:10:23.319454+0000","last_became_active":"2024-03-29T04:10:22.301559+0000","last_became_peered":"2024-03-29T04:10:22.301559+0000","last_unstale":"2024-03-29T04:10:23.319454+0000","last_undegraded":"2024-03-29T04:10:23.319454+0000","last_fullsized":"2024-03-29T04:10:23.319454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311930+0000","last_change":"2024-03-29T04:10:23.308697+0000","last_active":"2024-03-29T04:10:23.311930+0000","last_peered":"2024-03-29T04:10:23.311930+0000","last_clean":"2024-03-29T04:10:23.311930+0000","last_became_active":"2024-03-29T04:10:23.308430+0000","last_became_peered":"2024-03-29T04:10:23.308430+0000","last_unstale":"2024-03-29T04:10:23.311930+0000","last_undegraded":"2024-03-29T04:10:23.311930+0000","last_fullsized":"2024-03-29T04:10:23.311930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307389+0000","last_change":"2024-03-29T04:10:22.306797+0000","last_active":"2024-03-29T04:10:22.307389+0000","last_peered":"2024-03-29T04:10:22.307389+0000","last_clean":"2024-03-29T04:10:22.307389+0000","last_became_active":"2024-03-29T04:10:22.306504+0000","last_became_peered":"2024-03-29T04:10:22.306504+0000","last_unstale":"2024-03-29T04:10:22.307389+0000","last_undegraded":"2024-03-29T04:10:22.307389+0000","last_fullsized":"2024-03-29T04:10:22.307389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320184+0000","last_change":"2024-03-29T04:10:23.314380+0000","last_active":"2024-03-29T04:10:23.320184+0000","last_peered":"2024-03-29T04:10:23.320184+0000","last_clean":"2024-03-29T04:10:23.320184+0000","last_became_active":"2024-03-29T04:10:23.314129+0000","last_became_peered":"2024-03-29T04:10:23.314129+0000","last_unstale":"2024-03-29T04:10:23.320184+0000","last_undegraded":"2024-03-29T04:10:23.320184+0000","last_fullsized":"2024-03-29T04:10:23.320184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.311633+0000","last_change":"2024-03-29T04:10:22.310915+0000","last_active":"2024-03-29T04:10:22.311633+0000","last_peered":"2024-03-29T04:10:22.311633+0000","last_clean":"2024-03-29T04:10:22.311633+0000","last_became_active":"2024-03-29T04:10:22.310630+0000","last_became_peered":"2024-03-29T04:10:22.310630+0000","last_unstale":"2024-03-29T04:10:22.311633+0000","last_undegraded":"2024-03-29T04:10:22.311633+0000","last_fullsized":"2024-03-29T04:10:22.311633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301511+0000","last_change":"2024-03-29T04:10:22.301511+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301511+0000","last_undegraded":"2024-03-29T04:10:22.301511+0000","last_fullsized":"2024-03-29T04:10:22.301511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319869+0000","last_change":"2024-03-29T04:10:22.304262+0000","last_active":"2024-03-29T04:10:23.319869+0000","last_peered":"2024-03-29T04:10:23.319869+0000","last_clean":"2024-03-29T04:10:23.319869+0000","last_became_active":"2024-03-29T04:10:22.303999+0000","last_became_peered":"2024-03-29T04:10:22.303999+0000","last_unstale":"2024-03-29T04:10:23.319869+0000","last_undegraded":"2024-03-29T04:10:23.319869+0000","last_fullsized":"2024-03-29T04:10:23.319869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292420+0000","last_change":"2024-03-29T04:10:22.292420+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292420+0000","last_undegraded":"2024-03-29T04:10:22.292420+0000","last_fullsized":"2024-03-29T04:10:22.292420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295533+0000","last_change":"2024-03-29T04:10:22.307417+0000","last_active":"2024-03-29T04:10:23.295533+0000","last_peered":"2024-03-29T04:10:23.295533+0000","last_clean":"2024-03-29T04:10:23.295533+0000","last_became_active":"2024-03-29T04:10:22.307123+0000","last_became_peered":"2024-03-29T04:10:22.307123+0000","last_unstale":"2024-03-29T04:10:23.295533+0000","last_undegraded":"2024-03-29T04:10:23.295533+0000","last_fullsized":"2024-03-29T04:10:23.295533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312368+0000","last_change":"2024-03-29T04:10:23.308517+0000","last_active":"2024-03-29T04:10:23.312368+0000","last_peered":"2024-03-29T04:10:23.312368+0000","last_clean":"2024-03-29T04:10:23.312368+0000","last_became_active":"2024-03-29T04:10:23.308180+0000","last_became_peered":"2024-03-29T04:10:23.308180+0000","last_unstale":"2024-03-29T04:10:23.312368+0000","last_undegraded":"2024-03-29T04:10:23.312368+0000","last_fullsized":"2024-03-29T04:10:23.312368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.296007+0000","last_change":"2024-03-29T04:10:22.294851+0000","last_active":"2024-03-29T04:10:22.296007+0000","last_peered":"2024-03-29T04:10:22.296007+0000","last_clean":"2024-03-29T04:10:22.296007+0000","last_became_active":"2024-03-29T04:10:22.294580+0000","last_became_peered":"2024-03-29T04:10:22.294580+0000","last_unstale":"2024-03-29T04:10:22.296007+0000","last_undegraded":"2024-03-29T04:10:22.296007+0000","last_fullsized":"2024-03-29T04:10:22.296007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321112+0000","last_change":"2024-03-29T04:10:23.313712+0000","last_active":"2024-03-29T04:10:23.321112+0000","last_peered":"2024-03-29T04:10:23.321112+0000","last_clean":"2024-03-29T04:10:23.321112+0000","last_became_active":"2024-03-29T04:10:23.313467+0000","last_became_peered":"2024-03-29T04:10:23.313467+0000","last_unstale":"2024-03-29T04:10:23.321112+0000","last_undegraded":"2024-03-29T04:10:23.321112+0000","last_fullsized":"2024-03-29T04:10:23.321112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:39:13.669270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295738+0000","last_change":"2024-03-29T04:10:22.305312+0000","last_active":"2024-03-29T04:10:23.295738+0000","last_peered":"2024-03-29T04:10:23.295738+0000","last_clean":"2024-03-29T04:10:23.295738+0000","last_became_active":"2024-03-29T04:10:22.304984+0000","last_became_peered":"2024-03-29T04:10:22.304984+0000","last_unstale":"2024-03-29T04:10:23.295738+0000","last_undegraded":"2024-03-29T04:10:23.295738+0000","last_fullsized":"2024-03-29T04:10:23.295738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:35.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314641+0000","last_change":"2024-03-29T04:10:23.304934+0000","last_active":"2024-03-29T04:10:23.314641+0000","last_peered":"2024-03-29T04:10:23.314641+0000","last_clean":"2024-03-29T04:10:23.314641+0000","last_became_active":"2024-03-29T04:10:23.304669+0000","last_became_peered":"2024-03-29T04:10:23.304669+0000","last_unstale":"2024-03-29T04:10:23.314641+0000","last_undegraded":"2024-03-29T04:10:23.314641+0000","last_fullsized":"2024-03-29T04:10:23.314641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309574+0000","last_change":"2024-03-29T04:10:22.308050+0000","last_active":"2024-03-29T04:10:22.309574+0000","last_peered":"2024-03-29T04:10:22.309574+0000","last_clean":"2024-03-29T04:10:22.309574+0000","last_became_active":"2024-03-29T04:10:22.307753+0000","last_became_peered":"2024-03-29T04:10:22.307753+0000","last_unstale":"2024-03-29T04:10:22.309574+0000","last_undegraded":"2024-03-29T04:10:22.309574+0000","last_fullsized":"2024-03-29T04:10:22.309574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322352+0000","last_change":"2024-03-29T04:10:23.317220+0000","last_active":"2024-03-29T04:10:23.322352+0000","last_peered":"2024-03-29T04:10:23.322352+0000","last_clean":"2024-03-29T04:10:23.322352+0000","last_became_active":"2024-03-29T04:10:23.316962+0000","last_became_peered":"2024-03-29T04:10:23.316962+0000","last_unstale":"2024-03-29T04:10:23.322352+0000","last_undegraded":"2024-03-29T04:10:23.322352+0000","last_fullsized":"2024-03-29T04:10:23.322352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310684+0000","last_change":"2024-03-29T04:10:22.308995+0000","last_active":"2024-03-29T04:10:22.310684+0000","last_peered":"2024-03-29T04:10:22.310684+0000","last_clean":"2024-03-29T04:10:22.310684+0000","last_became_active":"2024-03-29T04:10:22.308679+0000","last_became_peered":"2024-03-29T04:10:22.308679+0000","last_unstale":"2024-03-29T04:10:22.310684+0000","last_undegraded":"2024-03-29T04:10:22.310684+0000","last_fullsized":"2024-03-29T04:10:22.310684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292102+0000","last_change":"2024-03-29T04:10:22.292102+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292102+0000","last_undegraded":"2024-03-29T04:10:22.292102+0000","last_fullsized":"2024-03-29T04:10:22.292102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323985+0000","last_change":"2024-03-29T04:10:22.306679+0000","last_active":"2024-03-29T04:10:23.323985+0000","last_peered":"2024-03-29T04:10:23.323985+0000","last_clean":"2024-03-29T04:10:23.323985+0000","last_became_active":"2024-03-29T04:10:22.306411+0000","last_became_peered":"2024-03-29T04:10:22.306411+0000","last_unstale":"2024-03-29T04:10:23.323985+0000","last_undegraded":"2024-03-29T04:10:23.323985+0000","last_fullsized":"2024-03-29T04:10:23.323985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:22.259732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322368+0000","last_change":"2024-03-29T04:10:23.316218+0000","last_active":"2024-03-29T04:10:23.322368+0000","last_peered":"2024-03-29T04:10:23.322368+0000","last_clean":"2024-03-29T04:10:23.322368+0000","last_became_active":"2024-03-29T04:10:23.315899+0000","last_became_peered":"2024-03-29T04:10:23.315899+0000","last_unstale":"2024-03-29T04:10:23.322368+0000","last_undegraded":"2024-03-29T04:10:23.322368+0000","last_fullsized":"2024-03-29T04:10:23.322368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322565+0000","last_change":"2024-03-29T04:10:22.310885+0000","last_active":"2024-03-29T04:10:23.322565+0000","last_peered":"2024-03-29T04:10:23.322565+0000","last_clean":"2024-03-29T04:10:23.322565+0000","last_became_active":"2024-03-29T04:10:22.310589+0000","last_became_peered":"2024-03-29T04:10:22.310589+0000","last_unstale":"2024-03-29T04:10:23.322565+0000","last_undegraded":"2024-03-29T04:10:23.322565+0000","last_fullsized":"2024-03-29T04:10:23.322565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:00.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317792+0000","last_change":"2024-03-29T04:10:23.312331+0000","last_active":"2024-03-29T04:10:23.317792+0000","last_peered":"2024-03-29T04:10:23.317792+0000","last_clean":"2024-03-29T04:10:23.317792+0000","last_became_active":"2024-03-29T04:10:23.311961+0000","last_became_peered":"2024-03-29T04:10:23.311961+0000","last_unstale":"2024-03-29T04:10:23.317792+0000","last_undegraded":"2024-03-29T04:10:23.317792+0000","last_fullsized":"2024-03-29T04:10:23.317792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.298233+0000","last_change":"2024-03-29T04:10:22.308119+0000","last_active":"2024-03-29T04:10:23.298233+0000","last_peered":"2024-03-29T04:10:23.298233+0000","last_clean":"2024-03-29T04:10:23.298233+0000","last_became_active":"2024-03-29T04:10:22.307849+0000","last_became_peered":"2024-03-29T04:10:22.307849+0000","last_unstale":"2024-03-29T04:10:23.298233+0000","last_undegraded":"2024-03-29T04:10:23.298233+0000","last_fullsized":"2024-03-29T04:10:23.298233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316478+0000","last_change":"2024-03-29T04:10:23.310446+0000","last_active":"2024-03-29T04:10:23.316478+0000","last_peered":"2024-03-29T04:10:23.316478+0000","last_clean":"2024-03-29T04:10:23.316478+0000","last_became_active":"2024-03-29T04:10:23.310183+0000","last_became_peered":"2024-03-29T04:10:23.310183+0000","last_unstale":"2024-03-29T04:10:23.316478+0000","last_undegraded":"2024-03-29T04:10:23.316478+0000","last_fullsized":"2024-03-29T04:10:23.316478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.291912+0000","last_change":"2024-03-29T04:10:22.297317+0000","last_active":"2024-03-29T04:10:23.291912+0000","last_peered":"2024-03-29T04:10:23.291912+0000","last_clean":"2024-03-29T04:10:23.291912+0000","last_became_active":"2024-03-29T04:10:22.297044+0000","last_became_peered":"2024-03-29T04:10:22.297044+0000","last_unstale":"2024-03-29T04:10:23.291912+0000","last_undegraded":"2024-03-29T04:10:23.291912+0000","last_fullsized":"2024-03-29T04:10:23.291912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:56.164692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317543+0000","last_change":"2024-03-29T04:10:23.312644+0000","last_active":"2024-03-29T04:10:23.317543+0000","last_peered":"2024-03-29T04:10:23.317543+0000","last_clean":"2024-03-29T04:10:23.317543+0000","last_became_active":"2024-03-29T04:10:23.312370+0000","last_became_peered":"2024-03-29T04:10:23.312370+0000","last_unstale":"2024-03-29T04:10:23.317543+0000","last_undegraded":"2024-03-29T04:10:23.317543+0000","last_fullsized":"2024-03-29T04:10:23.317543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320581+0000","last_change":"2024-03-29T04:10:22.304877+0000","last_active":"2024-03-29T04:10:23.320581+0000","last_peered":"2024-03-29T04:10:23.320581+0000","last_clean":"2024-03-29T04:10:23.320581+0000","last_became_active":"2024-03-29T04:10:22.304573+0000","last_became_peered":"2024-03-29T04:10:22.304573+0000","last_unstale":"2024-03-29T04:10:23.320581+0000","last_undegraded":"2024-03-29T04:10:23.320581+0000","last_fullsized":"2024-03-29T04:10:23.320581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.315960+0000","last_change":"2024-03-29T04:10:23.309896+0000","last_active":"2024-03-29T04:10:23.315960+0000","last_peered":"2024-03-29T04:10:23.315960+0000","last_clean":"2024-03-29T04:10:23.315960+0000","last_became_active":"2024-03-29T04:10:23.308057+0000","last_became_peered":"2024-03-29T04:10:23.308057+0000","last_unstale":"2024-03-29T04:10:23.315960+0000","last_undegraded":"2024-03-29T04:10:23.315960+0000","last_fullsized":"2024-03-29T04:10:23.315960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307929+0000","last_change":"2024-03-29T04:10:22.307155+0000","last_active":"2024-03-29T04:10:22.307929+0000","last_peered":"2024-03-29T04:10:22.307929+0000","last_clean":"2024-03-29T04:10:22.307929+0000","last_became_active":"2024-03-29T04:10:22.306817+0000","last_became_peered":"2024-03-29T04:10:22.306817+0000","last_unstale":"2024-03-29T04:10:22.307929+0000","last_undegraded":"2024-03-29T04:10:22.307929+0000","last_fullsized":"2024-03-29T04:10:22.307929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301030+0000","last_change":"2024-03-29T04:10:22.301030+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301030+0000","last_undegraded":"2024-03-29T04:10:22.301030+0000","last_fullsized":"2024-03-29T04:10:22.301030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309562+0000","last_change":"2024-03-29T04:10:22.307690+0000","last_active":"2024-03-29T04:10:22.309562+0000","last_peered":"2024-03-29T04:10:22.309562+0000","last_clean":"2024-03-29T04:10:22.309562+0000","last_became_active":"2024-03-29T04:10:22.307424+0000","last_became_peered":"2024-03-29T04:10:22.307424+0000","last_unstale":"2024-03-29T04:10:22.309562+0000","last_undegraded":"2024-03-29T04:10:22.309562+0000","last_fullsized":"2024-03-29T04:10:22.309562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292383+0000","last_change":"2024-03-29T04:10:22.292383+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292383+0000","last_undegraded":"2024-03-29T04:10:22.292383+0000","last_fullsized":"2024-03-29T04:10:22.292383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318361+0000","last_change":"2024-03-29T04:10:22.303744+0000","last_active":"2024-03-29T04:10:23.318361+0000","last_peered":"2024-03-29T04:10:23.318361+0000","last_clean":"2024-03-29T04:10:23.318361+0000","last_became_active":"2024-03-29T04:10:22.303477+0000","last_became_peered":"2024-03-29T04:10:22.303477+0000","last_unstale":"2024-03-29T04:10:23.318361+0000","last_undegraded":"2024-03-29T04:10:23.318361+0000","last_fullsized":"2024-03-29T04:10:23.318361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:03:07.972943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.310284+0000","last_change":"2024-03-29T04:10:23.306990+0000","last_active":"2024-03-29T04:10:23.310284+0000","last_peered":"2024-03-29T04:10:23.310284+0000","last_clean":"2024-03-29T04:10:23.310284+0000","last_became_active":"2024-03-29T04:10:23.306635+0000","last_became_peered":"2024-03-29T04:10:23.306635+0000","last_unstale":"2024-03-29T04:10:23.310284+0000","last_undegraded":"2024-03-29T04:10:23.310284+0000","last_fullsized":"2024-03-29T04:10:23.310284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:25.613005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297872+0000","last_change":"2024-03-29T04:10:22.300528+0000","last_active":"2024-03-29T04:10:23.297872+0000","last_peered":"2024-03-29T04:10:23.297872+0000","last_clean":"2024-03-29T04:10:23.297872+0000","last_became_active":"2024-03-29T04:10:22.299874+0000","last_became_peered":"2024-03-29T04:10:22.299874+0000","last_unstale":"2024-03-29T04:10:23.297872+0000","last_undegraded":"2024-03-29T04:10:23.297872+0000","last_fullsized":"2024-03-29T04:10:23.297872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316810+0000","last_change":"2024-03-29T04:10:23.311796+0000","last_active":"2024-03-29T04:10:23.316810+0000","last_peered":"2024-03-29T04:10:23.316810+0000","last_clean":"2024-03-29T04:10:23.316810+0000","last_became_active":"2024-03-29T04:10:23.311351+0000","last_became_peered":"2024-03-29T04:10:23.311351+0000","last_unstale":"2024-03-29T04:10:23.316810+0000","last_undegraded":"2024-03-29T04:10:23.316810+0000","last_fullsized":"2024-03-29T04:10:23.316810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309583+0000","last_change":"2024-03-29T04:10:22.308134+0000","last_active":"2024-03-29T04:10:23.309583+0000","last_peered":"2024-03-29T04:10:23.309583+0000","last_clean":"2024-03-29T04:10:23.309583+0000","last_became_active":"2024-03-29T04:10:22.307860+0000","last_became_peered":"2024-03-29T04:10:22.307860+0000","last_unstale":"2024-03-29T04:10:23.309583+0000","last_undegraded":"2024-03-29T04:10:23.309583+0000","last_fullsized":"2024-03-29T04:10:23.309583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:20:17.587318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311340+0000","last_change":"2024-03-29T04:10:23.309927+0000","last_active":"2024-03-29T04:10:23.311340+0000","last_peered":"2024-03-29T04:10:23.311340+0000","last_clean":"2024-03-29T04:10:23.311340+0000","last_became_active":"2024-03-29T04:10:23.309623+0000","last_became_peered":"2024-03-29T04:10:23.309623+0000","last_unstale":"2024-03-29T04:10:23.311340+0000","last_undegraded":"2024-03-29T04:10:23.311340+0000","last_fullsized":"2024-03-29T04:10:23.311340+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.301262+0000","last_change":"2024-03-29T04:10:22.297913+0000","last_active":"2024-03-29T04:10:23.301262+0000","last_peered":"2024-03-29T04:10:23.301262+0000","last_clean":"2024-03-29T04:10:23.301262+0000","last_became_active":"2024-03-29T04:10:22.296015+0000","last_became_peered":"2024-03-29T04:10:22.296015+0000","last_unstale":"2024-03-29T04:10:23.301262+0000","last_undegraded":"2024-03-29T04:10:23.301262+0000","last_fullsized":"2024-03-29T04:10:23.301262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:13.154992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300580+0000","last_change":"2024-03-29T04:10:22.300580+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300580+0000","last_undegraded":"2024-03-29T04:10:22.300580+0000","last_fullsized":"2024-03-29T04:10:22.300580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292907+0000","last_change":"2024-03-29T04:10:22.305287+0000","last_active":"2024-03-29T04:10:23.292907+0000","last_peered":"2024-03-29T04:10:23.292907+0000","last_clean":"2024-03-29T04:10:23.292907+0000","last_became_active":"2024-03-29T04:10:22.304999+0000","last_became_peered":"2024-03-29T04:10:22.304999+0000","last_unstale":"2024-03-29T04:10:23.292907+0000","last_undegraded":"2024-03-29T04:10:23.292907+0000","last_fullsized":"2024-03-29T04:10:23.292907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312347+0000","last_change":"2024-03-29T04:10:23.307638+0000","last_active":"2024-03-29T04:10:23.312347+0000","last_peered":"2024-03-29T04:10:23.312347+0000","last_clean":"2024-03-29T04:10:23.312347+0000","last_became_active":"2024-03-29T04:10:23.307381+0000","last_became_peered":"2024-03-29T04:10:23.307381+0000","last_unstale":"2024-03-29T04:10:23.312347+0000","last_undegraded":"2024-03-29T04:10:23.312347+0000","last_fullsized":"2024-03-29T04:10:23.312347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299841+0000","last_change":"2024-03-29T04:10:22.307254+0000","last_active":"2024-03-29T04:10:23.299841+0000","last_peered":"2024-03-29T04:10:23.299841+0000","last_clean":"2024-03-29T04:10:23.299841+0000","last_became_active":"2024-03-29T04:10:22.306976+0000","last_became_peered":"2024-03-29T04:10:22.306976+0000","last_unstale":"2024-03-29T04:10:23.299841+0000","last_undegraded":"2024-03-29T04:10:23.299841+0000","last_fullsized":"2024-03-29T04:10:23.299841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:54:11.970574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320128+0000","last_change":"2024-03-29T04:10:23.317609+0000","last_active":"2024-03-29T04:10:23.320128+0000","last_peered":"2024-03-29T04:10:23.320128+0000","last_clean":"2024-03-29T04:10:23.320128+0000","last_became_active":"2024-03-29T04:10:23.317364+0000","last_became_peered":"2024-03-29T04:10:23.317364+0000","last_unstale":"2024-03-29T04:10:23.320128+0000","last_undegraded":"2024-03-29T04:10:23.320128+0000","last_fullsized":"2024-03-29T04:10:23.320128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310663+0000","last_change":"2024-03-29T04:10:22.308920+0000","last_active":"2024-03-29T04:10:22.310663+0000","last_peered":"2024-03-29T04:10:22.310663+0000","last_clean":"2024-03-29T04:10:22.310663+0000","last_became_active":"2024-03-29T04:10:22.308648+0000","last_became_peered":"2024-03-29T04:10:22.308648+0000","last_unstale":"2024-03-29T04:10:22.310663+0000","last_undegraded":"2024-03-29T04:10:22.310663+0000","last_fullsized":"2024-03-29T04:10:22.310663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314664+0000","last_change":"2024-03-29T04:10:23.305533+0000","last_active":"2024-03-29T04:10:23.314664+0000","last_peered":"2024-03-29T04:10:23.314664+0000","last_clean":"2024-03-29T04:10:23.314664+0000","last_became_active":"2024-03-29T04:10:23.305295+0000","last_became_peered":"2024-03-29T04:10:23.305295+0000","last_unstale":"2024-03-29T04:10:23.314664+0000","last_undegraded":"2024-03-29T04:10:23.314664+0000","last_fullsized":"2024-03-29T04:10:23.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304196+0000","last_change":"2024-03-29T04:10:22.309267+0000","last_active":"2024-03-29T04:10:23.304196+0000","last_peered":"2024-03-29T04:10:23.304196+0000","last_clean":"2024-03-29T04:10:23.304196+0000","last_became_active":"2024-03-29T04:10:22.308996+0000","last_became_peered":"2024-03-29T04:10:22.308996+0000","last_unstale":"2024-03-29T04:10:23.304196+0000","last_undegraded":"2024-03-29T04:10:23.304196+0000","last_fullsized":"2024-03-29T04:10:23.304196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:27.354497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317732+0000","last_change":"2024-03-29T04:10:23.313551+0000","last_active":"2024-03-29T04:10:23.317732+0000","last_peered":"2024-03-29T04:10:23.317732+0000","last_clean":"2024-03-29T04:10:23.317732+0000","last_became_active":"2024-03-29T04:10:23.313289+0000","last_became_peered":"2024-03-29T04:10:23.313289+0000","last_unstale":"2024-03-29T04:10:23.317732+0000","last_undegraded":"2024-03-29T04:10:23.317732+0000","last_fullsized":"2024-03-29T04:10:23.317732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299143+0000","last_change":"2024-03-29T04:10:22.308648+0000","last_active":"2024-03-29T04:10:23.299143+0000","last_peered":"2024-03-29T04:10:23.299143+0000","last_clean":"2024-03-29T04:10:23.299143+0000","last_became_active":"2024-03-29T04:10:22.308334+0000","last_became_peered":"2024-03-29T04:10:22.308334+0000","last_unstale":"2024-03-29T04:10:23.299143+0000","last_undegraded":"2024-03-29T04:10:23.299143+0000","last_fullsized":"2024-03-29T04:10:23.299143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322380+0000","last_change":"2024-03-29T04:10:23.319502+0000","last_active":"2024-03-29T04:10:23.322380+0000","last_peered":"2024-03-29T04:10:23.322380+0000","last_clean":"2024-03-29T04:10:23.322380+0000","last_became_active":"2024-03-29T04:10:23.319209+0000","last_became_peered":"2024-03-29T04:10:23.319209+0000","last_unstale":"2024-03-29T04:10:23.322380+0000","last_undegraded":"2024-03-29T04:10:23.322380+0000","last_fullsized":"2024-03-29T04:10:23.322380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.306085+0000","last_change":"2024-03-29T04:10:22.307995+0000","last_active":"2024-03-29T04:10:23.306085+0000","last_peered":"2024-03-29T04:10:23.306085+0000","last_clean":"2024-03-29T04:10:23.306085+0000","last_became_active":"2024-03-29T04:10:22.307723+0000","last_became_peered":"2024-03-29T04:10:22.307723+0000","last_unstale":"2024-03-29T04:10:23.306085+0000","last_undegraded":"2024-03-29T04:10:23.306085+0000","last_fullsized":"2024-03-29T04:10:23.306085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316387+0000","last_change":"2024-03-29T04:10:23.308586+0000","last_active":"2024-03-29T04:10:23.316387+0000","last_peered":"2024-03-29T04:10:23.316387+0000","last_clean":"2024-03-29T04:10:23.316387+0000","last_became_active":"2024-03-29T04:10:23.308278+0000","last_became_peered":"2024-03-29T04:10:23.308278+0000","last_unstale":"2024-03-29T04:10:23.316387+0000","last_undegraded":"2024-03-29T04:10:23.316387+0000","last_fullsized":"2024-03-29T04:10:23.316387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295435+0000","last_change":"2024-03-29T04:10:22.309876+0000","last_active":"2024-03-29T04:10:23.295435+0000","last_peered":"2024-03-29T04:10:23.295435+0000","last_clean":"2024-03-29T04:10:23.295435+0000","last_became_active":"2024-03-29T04:10:22.309473+0000","last_became_peered":"2024-03-29T04:10:22.309473+0000","last_unstale":"2024-03-29T04:10:23.295435+0000","last_undegraded":"2024-03-29T04:10:23.295435+0000","last_fullsized":"2024-03-29T04:10:23.295435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:20.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292868+0000","last_change":"2024-03-29T04:10:22.292868+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292868+0000","last_undegraded":"2024-03-29T04:10:22.292868+0000","last_fullsized":"2024-03-29T04:10:22.292868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.303802+0000","last_change":"2024-03-29T04:10:22.302187+0000","last_active":"2024-03-29T04:10:22.303802+0000","last_peered":"2024-03-29T04:10:22.303802+0000","last_clean":"2024-03-29T04:10:22.303802+0000","last_became_active":"2024-03-29T04:10:22.301830+0000","last_became_peered":"2024-03-29T04:10:22.301830+0000","last_unstale":"2024-03-29T04:10:22.303802+0000","last_undegraded":"2024-03-29T04:10:22.303802+0000","last_fullsized":"2024-03-29T04:10:22.303802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314171+0000","last_change":"2024-03-29T04:10:23.303127+0000","last_active":"2024-03-29T04:10:23.314171+0000","last_peered":"2024-03-29T04:10:23.314171+0000","last_clean":"2024-03-29T04:10:23.314171+0000","last_became_active":"2024-03-29T04:10:23.302882+0000","last_became_peered":"2024-03-29T04:10:23.302882+0000","last_unstale":"2024-03-29T04:10:23.314171+0000","last_undegraded":"2024-03-29T04:10:23.314171+0000","last_fullsized":"2024-03-29T04:10:23.314171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297157+0000","last_change":"2024-03-29T04:10:22.304632+0000","last_active":"2024-03-29T04:10:23.297157+0000","last_peered":"2024-03-29T04:10:23.297157+0000","last_clean":"2024-03-29T04:10:23.297157+0000","last_became_active":"2024-03-29T04:10:22.304363+0000","last_became_peered":"2024-03-29T04:10:22.304363+0000","last_unstale":"2024-03-29T04:10:23.297157+0000","last_undegraded":"2024-03-29T04:10:23.297157+0000","last_fullsized":"2024-03-29T04:10:23.297157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:35.091291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301182+0000","last_change":"2024-03-29T04:10:22.301182+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301182+0000","last_undegraded":"2024-03-29T04:10:22.301182+0000","last_fullsized":"2024-03-29T04:10:22.301182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.293779+0000","last_change":"2024-03-29T04:10:22.310649+0000","last_active":"2024-03-29T04:10:23.293779+0000","last_peered":"2024-03-29T04:10:23.293779+0000","last_clean":"2024-03-29T04:10:23.293779+0000","last_became_active":"2024-03-29T04:10:22.310382+0000","last_became_peered":"2024-03-29T04:10:22.310382+0000","last_unstale":"2024-03-29T04:10:23.293779+0000","last_undegraded":"2024-03-29T04:10:23.293779+0000","last_fullsized":"2024-03-29T04:10:23.293779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292459+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:23.292459+0000","last_peered":"2024-03-29T04:10:23.292459+0000","last_clean":"2024-03-29T04:10:23.292459+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:23.292459+0000","last_undegraded":"2024-03-29T04:10:23.292459+0000","last_fullsized":"2024-03-29T04:10:23.292459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":192127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:26.828 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:26.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 Processor -- start 2024-03-29T04:10:26.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 -- start start 2024-03-29T04:10:26.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faa980d4a90 0x7faa980d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d36f0 con 0x7faa980d2f00 2024-03-29T04:10:26.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d39d0 con 0x7faa9815e950 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.003+0000 7faa9e6f8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faa980d3cb0 con 0x7faa980d4a90 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa9cc95640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faa980d2f00 0x7faa980b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54578/0 (socket says 172.21.15.181:54578) 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa9cc95640 1 -- 172.21.15.181:0/2911086154 learned_addr learned my addr 172.21.15.181:0/2911086154 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa977fe640 1 --2- 172.21.15.181:0/2911086154 >> v2:172.21.15.187:3300/0 conn(0x7faa980d4a90 0x7faa980d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541868343 0 0) 0x7faa980d36f0 con 0x7faa980d2f00 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa80003520 con 0x7faa980d2f00 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922498035 0 0) 0x7faa980d39d0 con 0x7faa9815e950 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa980d36f0 con 0x7faa9815e950 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa977fe640 1 -- 172.21.15.181:0/2911086154 >> v1:172.21.15.187:6789/0 conn(0x7faa9815e950 legacy=0x7faa980d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 967893356 0 0) 0x7faa80003520 con 0x7faa980d2f00 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa977fe640 1 -- 172.21.15.181:0/2911086154 >> v1:172.21.15.181:6789/0 conn(0x7faa980d2f00 legacy=0x7faa980b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.004+0000 7faa977fe640 1 -- 172.21.15.181:0/2911086154 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980b7ac0 con 0x7faa980d4a90 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.005+0000 7faa977fe640 1 --2- 172.21.15.181:0/2911086154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa980d4a90 0x7faa980d29c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0097a0 tx=0x7faa8c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab7a35d91b171f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.005+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900019f0 con 0x7faa980d4a90 2024-03-29T04:10:26.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.005+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c037070 con 0x7faa980d4a90 2024-03-29T04:10:26.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.005+0000 7faa96ffd640 1 -- 172.21.15.181:0/2911086154 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c0317b0 con 0x7faa980d4a90 2024-03-29T04:10:26.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2911086154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa980d4a90 msgr2=0x7faa980d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2911086154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa980d4a90 0x7faa980d29c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0097a0 tx=0x7faa8c02b8b0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2911086154 shutdown_connections 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2911086154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa980d4a90 0x7faa980d29c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2911086154 >> 172.21.15.181:0/2911086154 conn(0x7faa980676e0 msgr2=0x7faa98067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2911086154 shutdown_connections 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.006+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2911086154 wait complete. 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.007+0000 7faa9e6f8640 1 Processor -- start 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.007+0000 7faa9e6f8640 1 -- start start 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa980d2f00 0x7faa980abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa9815e950 0x7faa980b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa9815ed30 con 0x7faa980d4a90 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faa980d33b0 con 0x7faa9815e950 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9e6f8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faa9815eeb0 con 0x7faa980d2f00 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55150/0 (socket says 172.21.15.181:55150) 2024-03-29T04:10:26.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 -- 172.21.15.181:0/2140161348 learned_addr learned my addr 172.21.15.181:0/2140161348 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa9cc95640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa9815e950 0x7faa980b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa97fff640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa980d2f00 0x7faa980abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 -- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa980d2f00 msgr2=0x7faa980abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa980d2f00 0x7faa980abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 -- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa9815e950 msgr2=0x7faa980b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa9815e950 0x7faa980b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 -- 172.21.15.181:0/2140161348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981f53d0 con 0x7faa980d4a90 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.008+0000 7faa977fe640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02b8f0 tx=0x7faa8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.009+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c032620 con 0x7faa980d4a90 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.009+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c0327e0 con 0x7faa980d4a90 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.009+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa980ac1e0 con 0x7faa980d4a90 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.009+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c032bd0 con 0x7faa980d4a90 2024-03-29T04:10:26.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.009+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981585c0 con 0x7faa980d4a90 2024-03-29T04:10:26.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.010+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98075a10 con 0x7faa980d4a90 2024-03-29T04:10:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.010+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faa8c04c060 con 0x7faa980d4a90 2024-03-29T04:10:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.011+0000 7faa94ff9640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 0x7faa8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.011+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7faa8c0ad230 con 0x7faa980d4a90 2024-03-29T04:10:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.011+0000 7faa97fff640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 0x7faa8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:26.992 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.011+0000 7faa97fff640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 0x7faa8406e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa880011e0 tx=0x7faa88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:26.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.019+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faa8c040070 con 0x7faa980d4a90 2024-03-29T04:10:27.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.196+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa980ce110 con 0x7faa8406c530 2024-03-29T04:10:27.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.202+0000 7faa94ff9640 1 -- 172.21.15.181:0/2140161348 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250415 (secure 0 0 0) 0x7faa980ce110 con 0x7faa8406c530 2024-03-29T04:10:27.184 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:27.185 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 msgr2=0x7faa8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 0x7faa8406e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa880011e0 tx=0x7faa88009040 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 msgr2=0x7faa980b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02b8f0 tx=0x7faa8c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 shutdown_connections 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa9815e950 0x7faa980b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa980d4a90 0x7faa980b82c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faa8406c530 0x7faa8406e9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 --2- 172.21.15.181:0/2140161348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faa980d2f00 0x7faa980abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.205+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 >> 172.21.15.181:0/2140161348 conn(0x7faa980676e0 msgr2=0x7faa980d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.206+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 shutdown_connections 2024-03-29T04:10:27.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.206+0000 7faa9e6f8640 1 -- 172.21.15.181:0/2140161348 wait complete. 2024-03-29T04:10:27.199 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-03-29T04:10:25.587035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220684,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754036,"statfs":{"total":773094113280,"available":772868132864,"internally_reserved":0,"allocated":4616192,"data_stored":2649928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.305977"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318073+0000","last_change":"2024-03-29T04:10:23.313049+0000","last_active":"2024-03-29T04:10:23.318073+0000","last_peered":"2024-03-29T04:10:23.318073+0000","last_clean":"2024-03-29T04:10:23.318073+0000","last_became_active":"2024-03-29T04:10:23.312619+0000","last_became_peered":"2024-03-29T04:10:23.312619+0000","last_unstale":"2024-03-29T04:10:23.318073+0000","last_undegraded":"2024-03-29T04:10:23.318073+0000","last_fullsized":"2024-03-29T04:10:23.318073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309796+0000","last_change":"2024-03-29T04:10:23.304958+0000","last_active":"2024-03-29T04:10:23.309796+0000","last_peered":"2024-03-29T04:10:23.309796+0000","last_clean":"2024-03-29T04:10:23.309796+0000","last_became_active":"2024-03-29T04:10:23.304714+0000","last_became_peered":"2024-03-29T04:10:23.304714+0000","last_unstale":"2024-03-29T04:10:23.309796+0000","last_undegraded":"2024-03-29T04:10:23.309796+0000","last_fullsized":"2024-03-29T04:10:23.309796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322299+0000","last_change":"2024-03-29T04:10:23.315527+0000","last_active":"2024-03-29T04:10:23.322299+0000","last_peered":"2024-03-29T04:10:23.322299+0000","last_clean":"2024-03-29T04:10:23.322299+0000","last_became_active":"2024-03-29T04:10:23.315242+0000","last_became_peered":"2024-03-29T04:10:23.315242+0000","last_unstale":"2024-03-29T04:10:23.322299+0000","last_undegraded":"2024-03-29T04:10:23.322299+0000","last_fullsized":"2024-03-29T04:10:23.322299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321235+0000","last_change":"2024-03-29T04:10:23.315865+0000","last_active":"2024-03-29T04:10:23.321235+0000","last_peered":"2024-03-29T04:10:23.321235+0000","last_clean":"2024-03-29T04:10:23.321235+0000","last_became_active":"2024-03-29T04:10:23.315580+0000","last_became_peered":"2024-03-29T04:10:23.315580+0000","last_unstale":"2024-03-29T04:10:23.321235+0000","last_undegraded":"2024-03-29T04:10:23.321235+0000","last_fullsized":"2024-03-29T04:10:23.321235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:33:01.952778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321146+0000","last_change":"2024-03-29T04:10:23.315150+0000","last_active":"2024-03-29T04:10:23.321146+0000","last_peered":"2024-03-29T04:10:23.321146+0000","last_clean":"2024-03-29T04:10:23.321146+0000","last_became_active":"2024-03-29T04:10:23.314879+0000","last_became_peered":"2024-03-29T04:10:23.314879+0000","last_unstale":"2024-03-29T04:10:23.321146+0000","last_undegraded":"2024-03-29T04:10:23.321146+0000","last_fullsized":"2024-03-29T04:10:23.321146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.011645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316771+0000","last_change":"2024-03-29T04:10:23.311220+0000","last_active":"2024-03-29T04:10:23.316771+0000","last_peered":"2024-03-29T04:10:23.316771+0000","last_clean":"2024-03-29T04:10:23.316771+0000","last_became_active":"2024-03-29T04:10:23.310727+0000","last_became_peered":"2024-03-29T04:10:23.310727+0000","last_unstale":"2024-03-29T04:10:23.316771+0000","last_undegraded":"2024-03-29T04:10:23.316771+0000","last_fullsized":"2024-03-29T04:10:23.316771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.302395+0000","last_change":"2024-03-29T04:10:22.302395+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.302395+0000","last_undegraded":"2024-03-29T04:10:22.302395+0000","last_fullsized":"2024-03-29T04:10:22.302395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317690+0000","last_change":"2024-03-29T04:10:23.314124+0000","last_active":"2024-03-29T04:10:23.317690+0000","last_peered":"2024-03-29T04:10:23.317690+0000","last_clean":"2024-03-29T04:10:23.317690+0000","last_became_active":"2024-03-29T04:10:23.313434+0000","last_became_peered":"2024-03-29T04:10:23.313434+0000","last_unstale":"2024-03-29T04:10:23.317690+0000","last_undegraded":"2024-03-29T04:10:23.317690+0000","last_fullsized":"2024-03-29T04:10:23.317690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322272+0000","last_change":"2024-03-29T04:10:23.315461+0000","last_active":"2024-03-29T04:10:23.322272+0000","last_peered":"2024-03-29T04:10:23.322272+0000","last_clean":"2024-03-29T04:10:23.322272+0000","last_became_active":"2024-03-29T04:10:23.315163+0000","last_became_peered":"2024-03-29T04:10:23.315163+0000","last_unstale":"2024-03-29T04:10:23.322272+0000","last_undegraded":"2024-03-29T04:10:23.322272+0000","last_fullsized":"2024-03-29T04:10:23.322272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.303882+0000","last_change":"2024-03-29T04:10:22.303882+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.303882+0000","last_undegraded":"2024-03-29T04:10:22.303882+0000","last_fullsized":"2024-03-29T04:10:22.303882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301429+0000","last_change":"2024-03-29T04:10:22.301429+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301429+0000","last_undegraded":"2024-03-29T04:10:22.301429+0000","last_fullsized":"2024-03-29T04:10:22.301429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311095+0000","last_change":"2024-03-29T04:10:23.309450+0000","last_active":"2024-03-29T04:10:23.311095+0000","last_peered":"2024-03-29T04:10:23.311095+0000","last_clean":"2024-03-29T04:10:23.311095+0000","last_became_active":"2024-03-29T04:10:23.309186+0000","last_became_peered":"2024-03-29T04:10:23.309186+0000","last_unstale":"2024-03-29T04:10:23.311095+0000","last_undegraded":"2024-03-29T04:10:23.311095+0000","last_fullsized":"2024-03-29T04:10:23.311095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.135609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300951+0000","last_change":"2024-03-29T04:10:22.300951+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300951+0000","last_undegraded":"2024-03-29T04:10:22.300951+0000","last_fullsized":"2024-03-29T04:10:22.300951+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.293775+0000","last_change":"2024-03-29T04:10:22.293775+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.293775+0000","last_undegraded":"2024-03-29T04:10:22.293775+0000","last_fullsized":"2024-03-29T04:10:22.293775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301942+0000","last_change":"2024-03-29T04:10:22.301942+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301942+0000","last_undegraded":"2024-03-29T04:10:22.301942+0000","last_fullsized":"2024-03-29T04:10:22.301942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316831+0000","last_change":"2024-03-29T04:10:23.311958+0000","last_active":"2024-03-29T04:10:23.316831+0000","last_peered":"2024-03-29T04:10:23.316831+0000","last_clean":"2024-03-29T04:10:23.316831+0000","last_became_active":"2024-03-29T04:10:23.311700+0000","last_became_peered":"2024-03-29T04:10:23.311700+0000","last_unstale":"2024-03-29T04:10:23.316831+0000","last_undegraded":"2024-03-29T04:10:23.316831+0000","last_fullsized":"2024-03-29T04:10:23.316831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316930+0000","last_change":"2024-03-29T04:10:23.313386+0000","last_active":"2024-03-29T04:10:23.316930+0000","last_peered":"2024-03-29T04:10:23.316930+0000","last_clean":"2024-03-29T04:10:23.316930+0000","last_became_active":"2024-03-29T04:10:23.313083+0000","last_became_peered":"2024-03-29T04:10:23.313083+0000","last_unstale":"2024-03-29T04:10:23.316930+0000","last_undegraded":"2024-03-29T04:10:23.316930+0000","last_fullsized":"2024-03-29T04:10:23.316930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304763+0000","last_change":"2024-03-29T04:10:23.300122+0000","last_active":"2024-03-29T04:10:23.304763+0000","last_peered":"2024-03-29T04:10:23.304763+0000","last_clean":"2024-03-29T04:10:23.304763+0000","last_became_active":"2024-03-29T04:10:23.299864+0000","last_became_peered":"2024-03-29T04:10:23.299864+0000","last_unstale":"2024-03-29T04:10:23.304763+0000","last_undegraded":"2024-03-29T04:10:23.304763+0000","last_fullsized":"2024-03-29T04:10:23.304763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:29.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321486+0000","last_change":"2024-03-29T04:10:23.316162+0000","last_active":"2024-03-29T04:10:23.321486+0000","last_peered":"2024-03-29T04:10:23.321486+0000","last_clean":"2024-03-29T04:10:23.321486+0000","last_became_active":"2024-03-29T04:10:23.315894+0000","last_became_peered":"2024-03-29T04:10:23.315894+0000","last_unstale":"2024-03-29T04:10:23.321486+0000","last_undegraded":"2024-03-29T04:10:23.321486+0000","last_fullsized":"2024-03-29T04:10:23.321486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:37.304758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.305943+0000","last_change":"2024-03-29T04:10:23.297888+0000","last_active":"2024-03-29T04:10:23.305943+0000","last_peered":"2024-03-29T04:10:23.305943+0000","last_clean":"2024-03-29T04:10:23.305943+0000","last_became_active":"2024-03-29T04:10:23.297611+0000","last_became_peered":"2024-03-29T04:10:23.297611+0000","last_unstale":"2024-03-29T04:10:23.305943+0000","last_undegraded":"2024-03-29T04:10:23.305943+0000","last_fullsized":"2024-03-29T04:10:23.305943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:51:35.316362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300543+0000","last_change":"2024-03-29T04:10:22.300543+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300543+0000","last_undegraded":"2024-03-29T04:10:22.300543+0000","last_fullsized":"2024-03-29T04:10:22.300543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291376+0000","last_change":"2024-03-29T04:10:22.291376+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291376+0000","last_undegraded":"2024-03-29T04:10:22.291376+0000","last_fullsized":"2024-03-29T04:10:22.291376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317436+0000","last_change":"2024-03-29T04:10:23.315012+0000","last_active":"2024-03-29T04:10:23.317436+0000","last_peered":"2024-03-29T04:10:23.317436+0000","last_clean":"2024-03-29T04:10:23.317436+0000","last_became_active":"2024-03-29T04:10:23.312553+0000","last_became_peered":"2024-03-29T04:10:23.312553+0000","last_unstale":"2024-03-29T04:10:23.317436+0000","last_undegraded":"2024-03-29T04:10:23.317436+0000","last_fullsized":"2024-03-29T04:10:23.317436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:36.093598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322166+0000","last_change":"2024-03-29T04:10:23.318761+0000","last_active":"2024-03-29T04:10:23.322166+0000","last_peered":"2024-03-29T04:10:23.322166+0000","last_clean":"2024-03-29T04:10:23.322166+0000","last_became_active":"2024-03-29T04:10:23.318414+0000","last_became_peered":"2024-03-29T04:10:23.318414+0000","last_unstale":"2024-03-29T04:10:23.322166+0000","last_undegraded":"2024-03-29T04:10:23.322166+0000","last_fullsized":"2024-03-29T04:10:23.322166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:40.265277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319387+0000","last_change":"2024-03-29T04:10:23.312068+0000","last_active":"2024-03-29T04:10:23.319387+0000","last_peered":"2024-03-29T04:10:23.319387+0000","last_clean":"2024-03-29T04:10:23.319387+0000","last_became_active":"2024-03-29T04:10:23.311776+0000","last_became_peered":"2024-03-29T04:10:23.311776+0000","last_unstale":"2024-03-29T04:10:23.319387+0000","last_undegraded":"2024-03-29T04:10:23.319387+0000","last_fullsized":"2024-03-29T04:10:23.319387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:59.921229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320160+0000","last_change":"2024-03-29T04:10:23.318246+0000","last_active":"2024-03-29T04:10:23.320160+0000","last_peered":"2024-03-29T04:10:23.320160+0000","last_clean":"2024-03-29T04:10:23.320160+0000","last_became_active":"2024-03-29T04:10:23.317989+0000","last_became_peered":"2024-03-29T04:10:23.317989+0000","last_unstale":"2024-03-29T04:10:23.320160+0000","last_undegraded":"2024-03-29T04:10:23.320160+0000","last_fullsized":"2024-03-29T04:10:23.320160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309790+0000","last_change":"2024-03-29T04:10:23.308341+0000","last_active":"2024-03-29T04:10:23.309790+0000","last_peered":"2024-03-29T04:10:23.309790+0000","last_clean":"2024-03-29T04:10:23.309790+0000","last_became_active":"2024-03-29T04:10:23.307789+0000","last_became_peered":"2024-03-29T04:10:23.307789+0000","last_unstale":"2024-03-29T04:10:23.309790+0000","last_undegraded":"2024-03-29T04:10:23.309790+0000","last_fullsized":"2024-03-29T04:10:23.309790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:08.599452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.299555+0000","last_change":"2024-03-29T04:10:22.299555+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.299555+0000","last_undegraded":"2024-03-29T04:10:22.299555+0000","last_fullsized":"2024-03-29T04:10:22.299555+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308971+0000","last_change":"2024-03-29T04:10:23.307876+0000","last_active":"2024-03-29T04:10:23.308971+0000","last_peered":"2024-03-29T04:10:23.308971+0000","last_clean":"2024-03-29T04:10:23.308971+0000","last_became_active":"2024-03-29T04:10:23.307565+0000","last_became_peered":"2024-03-29T04:10:23.307565+0000","last_unstale":"2024-03-29T04:10:23.308971+0000","last_undegraded":"2024-03-29T04:10:23.308971+0000","last_fullsized":"2024-03-29T04:10:23.308971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:11.709686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308760+0000","last_change":"2024-03-29T04:10:23.306970+0000","last_active":"2024-03-29T04:10:23.308760+0000","last_peered":"2024-03-29T04:10:23.308760+0000","last_clean":"2024-03-29T04:10:23.308760+0000","last_became_active":"2024-03-29T04:10:23.306708+0000","last_became_peered":"2024-03-29T04:10:23.306708+0000","last_unstale":"2024-03-29T04:10:23.308760+0000","last_undegraded":"2024-03-29T04:10:23.308760+0000","last_fullsized":"2024-03-29T04:10:23.308760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:15.611882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317417+0000","last_change":"2024-03-29T04:10:23.314775+0000","last_active":"2024-03-29T04:10:23.317417+0000","last_peered":"2024-03-29T04:10:23.317417+0000","last_clean":"2024-03-29T04:10:23.317417+0000","last_became_active":"2024-03-29T04:10:23.312160+0000","last_became_peered":"2024-03-29T04:10:23.312160+0000","last_unstale":"2024-03-29T04:10:23.317417+0000","last_undegraded":"2024-03-29T04:10:23.317417+0000","last_fullsized":"2024-03-29T04:10:23.317417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.313388+0000","last_change":"2024-03-29T04:10:23.304939+0000","last_active":"2024-03-29T04:10:23.313388+0000","last_peered":"2024-03-29T04:10:23.313388+0000","last_clean":"2024-03-29T04:10:23.313388+0000","last_became_active":"2024-03-29T04:10:23.304682+0000","last_became_peered":"2024-03-29T04:10:23.304682+0000","last_unstale":"2024-03-29T04:10:23.313388+0000","last_undegraded":"2024-03-29T04:10:23.313388+0000","last_fullsized":"2024-03-29T04:10:23.313388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323623+0000","last_change":"2024-03-29T04:10:23.318116+0000","last_active":"2024-03-29T04:10:23.323623+0000","last_peered":"2024-03-29T04:10:23.323623+0000","last_clean":"2024-03-29T04:10:23.323623+0000","last_became_active":"2024-03-29T04:10:23.317859+0000","last_became_peered":"2024-03-29T04:10:23.317859+0000","last_unstale":"2024-03-29T04:10:23.323623+0000","last_undegraded":"2024-03-29T04:10:23.323623+0000","last_fullsized":"2024-03-29T04:10:23.323623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319317+0000","last_change":"2024-03-29T04:10:22.297429+0000","last_active":"2024-03-29T04:10:23.319317+0000","last_peered":"2024-03-29T04:10:23.319317+0000","last_clean":"2024-03-29T04:10:23.319317+0000","last_became_active":"2024-03-29T04:10:22.297077+0000","last_became_peered":"2024-03-29T04:10:22.297077+0000","last_unstale":"2024-03-29T04:10:23.319317+0000","last_undegraded":"2024-03-29T04:10:23.319317+0000","last_fullsized":"2024-03-29T04:10:23.319317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:10.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320194+0000","last_change":"2024-03-29T04:10:23.316949+0000","last_active":"2024-03-29T04:10:23.320194+0000","last_peered":"2024-03-29T04:10:23.320194+0000","last_clean":"2024-03-29T04:10:23.320194+0000","last_became_active":"2024-03-29T04:10:23.316665+0000","last_became_peered":"2024-03-29T04:10:23.316665+0000","last_unstale":"2024-03-29T04:10:23.320194+0000","last_undegraded":"2024-03-29T04:10:23.320194+0000","last_fullsized":"2024-03-29T04:10:23.320194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316490+0000","last_change":"2024-03-29T04:10:22.307974+0000","last_active":"2024-03-29T04:10:23.316490+0000","last_peered":"2024-03-29T04:10:23.316490+0000","last_clean":"2024-03-29T04:10:23.316490+0000","last_became_active":"2024-03-29T04:10:22.306547+0000","last_became_peered":"2024-03-29T04:10:22.306547+0000","last_unstale":"2024-03-29T04:10:23.316490+0000","last_undegraded":"2024-03-29T04:10:23.316490+0000","last_fullsized":"2024-03-29T04:10:23.316490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291935+0000","last_change":"2024-03-29T04:10:22.291935+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291935+0000","last_undegraded":"2024-03-29T04:10:22.291935+0000","last_fullsized":"2024-03-29T04:10:22.291935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319454+0000","last_change":"2024-03-29T04:10:22.301842+0000","last_active":"2024-03-29T04:10:23.319454+0000","last_peered":"2024-03-29T04:10:23.319454+0000","last_clean":"2024-03-29T04:10:23.319454+0000","last_became_active":"2024-03-29T04:10:22.301559+0000","last_became_peered":"2024-03-29T04:10:22.301559+0000","last_unstale":"2024-03-29T04:10:23.319454+0000","last_undegraded":"2024-03-29T04:10:23.319454+0000","last_fullsized":"2024-03-29T04:10:23.319454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311930+0000","last_change":"2024-03-29T04:10:23.308697+0000","last_active":"2024-03-29T04:10:23.311930+0000","last_peered":"2024-03-29T04:10:23.311930+0000","last_clean":"2024-03-29T04:10:23.311930+0000","last_became_active":"2024-03-29T04:10:23.308430+0000","last_became_peered":"2024-03-29T04:10:23.308430+0000","last_unstale":"2024-03-29T04:10:23.311930+0000","last_undegraded":"2024-03-29T04:10:23.311930+0000","last_fullsized":"2024-03-29T04:10:23.311930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307389+0000","last_change":"2024-03-29T04:10:22.306797+0000","last_active":"2024-03-29T04:10:22.307389+0000","last_peered":"2024-03-29T04:10:22.307389+0000","last_clean":"2024-03-29T04:10:22.307389+0000","last_became_active":"2024-03-29T04:10:22.306504+0000","last_became_peered":"2024-03-29T04:10:22.306504+0000","last_unstale":"2024-03-29T04:10:22.307389+0000","last_undegraded":"2024-03-29T04:10:22.307389+0000","last_fullsized":"2024-03-29T04:10:22.307389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320184+0000","last_change":"2024-03-29T04:10:23.314380+0000","last_active":"2024-03-29T04:10:23.320184+0000","last_peered":"2024-03-29T04:10:23.320184+0000","last_clean":"2024-03-29T04:10:23.320184+0000","last_became_active":"2024-03-29T04:10:23.314129+0000","last_became_peered":"2024-03-29T04:10:23.314129+0000","last_unstale":"2024-03-29T04:10:23.320184+0000","last_undegraded":"2024-03-29T04:10:23.320184+0000","last_fullsized":"2024-03-29T04:10:23.320184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.311633+0000","last_change":"2024-03-29T04:10:22.310915+0000","last_active":"2024-03-29T04:10:22.311633+0000","last_peered":"2024-03-29T04:10:22.311633+0000","last_clean":"2024-03-29T04:10:22.311633+0000","last_became_active":"2024-03-29T04:10:22.310630+0000","last_became_peered":"2024-03-29T04:10:22.310630+0000","last_unstale":"2024-03-29T04:10:22.311633+0000","last_undegraded":"2024-03-29T04:10:22.311633+0000","last_fullsized":"2024-03-29T04:10:22.311633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301511+0000","last_change":"2024-03-29T04:10:22.301511+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301511+0000","last_undegraded":"2024-03-29T04:10:22.301511+0000","last_fullsized":"2024-03-29T04:10:22.301511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319869+0000","last_change":"2024-03-29T04:10:22.304262+0000","last_active":"2024-03-29T04:10:23.319869+0000","last_peered":"2024-03-29T04:10:23.319869+0000","last_clean":"2024-03-29T04:10:23.319869+0000","last_became_active":"2024-03-29T04:10:22.303999+0000","last_became_peered":"2024-03-29T04:10:22.303999+0000","last_unstale":"2024-03-29T04:10:23.319869+0000","last_undegraded":"2024-03-29T04:10:23.319869+0000","last_fullsized":"2024-03-29T04:10:23.319869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292420+0000","last_change":"2024-03-29T04:10:22.292420+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292420+0000","last_undegraded":"2024-03-29T04:10:22.292420+0000","last_fullsized":"2024-03-29T04:10:22.292420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295533+0000","last_change":"2024-03-29T04:10:22.307417+0000","last_active":"2024-03-29T04:10:23.295533+0000","last_peered":"2024-03-29T04:10:23.295533+0000","last_clean":"2024-03-29T04:10:23.295533+0000","last_became_active":"2024-03-29T04:10:22.307123+0000","last_became_peered":"2024-03-29T04:10:22.307123+0000","last_unstale":"2024-03-29T04:10:23.295533+0000","last_undegraded":"2024-03-29T04:10:23.295533+0000","last_fullsized":"2024-03-29T04:10:23.295533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312368+0000","last_change":"2024-03-29T04:10:23.308517+0000","last_active":"2024-03-29T04:10:23.312368+0000","last_peered":"2024-03-29T04:10:23.312368+0000","last_clean":"2024-03-29T04:10:23.312368+0000","last_became_active":"2024-03-29T04:10:23.308180+0000","last_became_peered":"2024-03-29T04:10:23.308180+0000","last_unstale":"2024-03-29T04:10:23.312368+0000","last_undegraded":"2024-03-29T04:10:23.312368+0000","last_fullsized":"2024-03-29T04:10:23.312368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.296007+0000","last_change":"2024-03-29T04:10:22.294851+0000","last_active":"2024-03-29T04:10:22.296007+0000","last_peered":"2024-03-29T04:10:22.296007+0000","last_clean":"2024-03-29T04:10:22.296007+0000","last_became_active":"2024-03-29T04:10:22.294580+0000","last_became_peered":"2024-03-29T04:10:22.294580+0000","last_unstale":"2024-03-29T04:10:22.296007+0000","last_undegraded":"2024-03-29T04:10:22.296007+0000","last_fullsized":"2024-03-29T04:10:22.296007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321112+0000","last_change":"2024-03-29T04:10:23.313712+0000","last_active":"2024-03-29T04:10:23.321112+0000","last_peered":"2024-03-29T04:10:23.321112+0000","last_clean":"2024-03-29T04:10:23.321112+0000","last_became_active":"2024-03-29T04:10:23.313467+0000","last_became_peered":"2024-03-29T04:10:23.313467+0000","last_unstale":"2024-03-29T04:10:23.321112+0000","last_undegraded":"2024-03-29T04:10:23.321112+0000","last_fullsized":"2024-03-29T04:10:23.321112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:39:13.669270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295738+0000","last_change":"2024-03-29T04:10:22.305312+0000","last_active":"2024-03-29T04:10:23.295738+0000","last_peered":"2024-03-29T04:10:23.295738+0000","last_clean":"2024-03-29T04:10:23.295738+0000","last_became_active":"2024-03-29T04:10:22.304984+0000","last_became_peered":"2024-03-29T04:10:22.304984+0000","last_unstale":"2024-03-29T04:10:23.295738+0000","last_undegraded":"2024-03-29T04:10:23.295738+0000","last_fullsized":"2024-03-29T04:10:23.295738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:35.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314641+0000","last_change":"2024-03-29T04:10:23.304934+0000","last_active":"2024-03-29T04:10:23.314641+0000","last_peered":"2024-03-29T04:10:23.314641+0000","last_clean":"2024-03-29T04:10:23.314641+0000","last_became_active":"2024-03-29T04:10:23.304669+0000","last_became_peered":"2024-03-29T04:10:23.304669+0000","last_unstale":"2024-03-29T04:10:23.314641+0000","last_undegraded":"2024-03-29T04:10:23.314641+0000","last_fullsized":"2024-03-29T04:10:23.314641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309574+0000","last_change":"2024-03-29T04:10:22.308050+0000","last_active":"2024-03-29T04:10:22.309574+0000","last_peered":"2024-03-29T04:10:22.309574+0000","last_clean":"2024-03-29T04:10:22.309574+0000","last_became_active":"2024-03-29T04:10:22.307753+0000","last_became_peered":"2024-03-29T04:10:22.307753+0000","last_unstale":"2024-03-29T04:10:22.309574+0000","last_undegraded":"2024-03-29T04:10:22.309574+0000","last_fullsized":"2024-03-29T04:10:22.309574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322352+0000","last_change":"2024-03-29T04:10:23.317220+0000","last_active":"2024-03-29T04:10:23.322352+0000","last_peered":"2024-03-29T04:10:23.322352+0000","last_clean":"2024-03-29T04:10:23.322352+0000","last_became_active":"2024-03-29T04:10:23.316962+0000","last_became_peered":"2024-03-29T04:10:23.316962+0000","last_unstale":"2024-03-29T04:10:23.322352+0000","last_undegraded":"2024-03-29T04:10:23.322352+0000","last_fullsized":"2024-03-29T04:10:23.322352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310684+0000","last_change":"2024-03-29T04:10:22.308995+0000","last_active":"2024-03-29T04:10:22.310684+0000","last_peered":"2024-03-29T04:10:22.310684+0000","last_clean":"2024-03-29T04:10:22.310684+0000","last_became_active":"2024-03-29T04:10:22.308679+0000","last_became_peered":"2024-03-29T04:10:22.308679+0000","last_unstale":"2024-03-29T04:10:22.310684+0000","last_undegraded":"2024-03-29T04:10:22.310684+0000","last_fullsized":"2024-03-29T04:10:22.310684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292102+0000","last_change":"2024-03-29T04:10:22.292102+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292102+0000","last_undegraded":"2024-03-29T04:10:22.292102+0000","last_fullsized":"2024-03-29T04:10:22.292102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323985+0000","last_change":"2024-03-29T04:10:22.306679+0000","last_active":"2024-03-29T04:10:23.323985+0000","last_peered":"2024-03-29T04:10:23.323985+0000","last_clean":"2024-03-29T04:10:23.323985+0000","last_became_active":"2024-03-29T04:10:22.306411+0000","last_became_peered":"2024-03-29T04:10:22.306411+0000","last_unstale":"2024-03-29T04:10:23.323985+0000","last_undegraded":"2024-03-29T04:10:23.323985+0000","last_fullsized":"2024-03-29T04:10:23.323985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:22.259732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322368+0000","last_change":"2024-03-29T04:10:23.316218+0000","last_active":"2024-03-29T04:10:23.322368+0000","last_peered":"2024-03-29T04:10:23.322368+0000","last_clean":"2024-03-29T04:10:23.322368+0000","last_became_active":"2024-03-29T04:10:23.315899+0000","last_became_peered":"2024-03-29T04:10:23.315899+0000","last_unstale":"2024-03-29T04:10:23.322368+0000","last_undegraded":"2024-03-29T04:10:23.322368+0000","last_fullsized":"2024-03-29T04:10:23.322368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322565+0000","last_change":"2024-03-29T04:10:22.310885+0000","last_active":"2024-03-29T04:10:23.322565+0000","last_peered":"2024-03-29T04:10:23.322565+0000","last_clean":"2024-03-29T04:10:23.322565+0000","last_became_active":"2024-03-29T04:10:22.310589+0000","last_became_peered":"2024-03-29T04:10:22.310589+0000","last_unstale":"2024-03-29T04:10:23.322565+0000","last_undegraded":"2024-03-29T04:10:23.322565+0000","last_fullsized":"2024-03-29T04:10:23.322565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:00.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317792+0000","last_change":"2024-03-29T04:10:23.312331+0000","last_active":"2024-03-29T04:10:23.317792+0000","last_peered":"2024-03-29T04:10:23.317792+0000","last_clean":"2024-03-29T04:10:23.317792+0000","last_became_active":"2024-03-29T04:10:23.311961+0000","last_became_peered":"2024-03-29T04:10:23.311961+0000","last_unstale":"2024-03-29T04:10:23.317792+0000","last_undegraded":"2024-03-29T04:10:23.317792+0000","last_fullsized":"2024-03-29T04:10:23.317792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.298233+0000","last_change":"2024-03-29T04:10:22.308119+0000","last_active":"2024-03-29T04:10:23.298233+0000","last_peered":"2024-03-29T04:10:23.298233+0000","last_clean":"2024-03-29T04:10:23.298233+0000","last_became_active":"2024-03-29T04:10:22.307849+0000","last_became_peered":"2024-03-29T04:10:22.307849+0000","last_unstale":"2024-03-29T04:10:23.298233+0000","last_undegraded":"2024-03-29T04:10:23.298233+0000","last_fullsized":"2024-03-29T04:10:23.298233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316478+0000","last_change":"2024-03-29T04:10:23.310446+0000","last_active":"2024-03-29T04:10:23.316478+0000","last_peered":"2024-03-29T04:10:23.316478+0000","last_clean":"2024-03-29T04:10:23.316478+0000","last_became_active":"2024-03-29T04:10:23.310183+0000","last_became_peered":"2024-03-29T04:10:23.310183+0000","last_unstale":"2024-03-29T04:10:23.316478+0000","last_undegraded":"2024-03-29T04:10:23.316478+0000","last_fullsized":"2024-03-29T04:10:23.316478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.291912+0000","last_change":"2024-03-29T04:10:22.297317+0000","last_active":"2024-03-29T04:10:23.291912+0000","last_peered":"2024-03-29T04:10:23.291912+0000","last_clean":"2024-03-29T04:10:23.291912+0000","last_became_active":"2024-03-29T04:10:22.297044+0000","last_became_peered":"2024-03-29T04:10:22.297044+0000","last_unstale":"2024-03-29T04:10:23.291912+0000","last_undegraded":"2024-03-29T04:10:23.291912+0000","last_fullsized":"2024-03-29T04:10:23.291912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:56.164692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317543+0000","last_change":"2024-03-29T04:10:23.312644+0000","last_active":"2024-03-29T04:10:23.317543+0000","last_peered":"2024-03-29T04:10:23.317543+0000","last_clean":"2024-03-29T04:10:23.317543+0000","last_became_active":"2024-03-29T04:10:23.312370+0000","last_became_peered":"2024-03-29T04:10:23.312370+0000","last_unstale":"2024-03-29T04:10:23.317543+0000","last_undegraded":"2024-03-29T04:10:23.317543+0000","last_fullsized":"2024-03-29T04:10:23.317543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320581+0000","last_change":"2024-03-29T04:10:22.304877+0000","last_active":"2024-03-29T04:10:23.320581+0000","last_peered":"2024-03-29T04:10:23.320581+0000","last_clean":"2024-03-29T04:10:23.320581+0000","last_became_active":"2024-03-29T04:10:22.304573+0000","last_became_peered":"2024-03-29T04:10:22.304573+0000","last_unstale":"2024-03-29T04:10:23.320581+0000","last_undegraded":"2024-03-29T04:10:23.320581+0000","last_fullsized":"2024-03-29T04:10:23.320581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.315960+0000","last_change":"2024-03-29T04:10:23.309896+0000","last_active":"2024-03-29T04:10:23.315960+0000","last_peered":"2024-03-29T04:10:23.315960+0000","last_clean":"2024-03-29T04:10:23.315960+0000","last_became_active":"2024-03-29T04:10:23.308057+0000","last_became_peered":"2024-03-29T04:10:23.308057+0000","last_unstale":"2024-03-29T04:10:23.315960+0000","last_undegraded":"2024-03-29T04:10:23.315960+0000","last_fullsized":"2024-03-29T04:10:23.315960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307929+0000","last_change":"2024-03-29T04:10:22.307155+0000","last_active":"2024-03-29T04:10:22.307929+0000","last_peered":"2024-03-29T04:10:22.307929+0000","last_clean":"2024-03-29T04:10:22.307929+0000","last_became_active":"2024-03-29T04:10:22.306817+0000","last_became_peered":"2024-03-29T04:10:22.306817+0000","last_unstale":"2024-03-29T04:10:22.307929+0000","last_undegraded":"2024-03-29T04:10:22.307929+0000","last_fullsized":"2024-03-29T04:10:22.307929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301030+0000","last_change":"2024-03-29T04:10:22.301030+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301030+0000","last_undegraded":"2024-03-29T04:10:22.301030+0000","last_fullsized":"2024-03-29T04:10:22.301030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309562+0000","last_change":"2024-03-29T04:10:22.307690+0000","last_active":"2024-03-29T04:10:22.309562+0000","last_peered":"2024-03-29T04:10:22.309562+0000","last_clean":"2024-03-29T04:10:22.309562+0000","last_became_active":"2024-03-29T04:10:22.307424+0000","last_became_peered":"2024-03-29T04:10:22.307424+0000","last_unstale":"2024-03-29T04:10:22.309562+0000","last_undegraded":"2024-03-29T04:10:22.309562+0000","last_fullsized":"2024-03-29T04:10:22.309562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292383+0000","last_change":"2024-03-29T04:10:22.292383+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292383+0000","last_undegraded":"2024-03-29T04:10:22.292383+0000","last_fullsized":"2024-03-29T04:10:22.292383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318361+0000","last_change":"2024-03-29T04:10:22.303744+0000","last_active":"2024-03-29T04:10:23.318361+0000","last_peered":"2024-03-29T04:10:23.318361+0000","last_clean":"2024-03-29T04:10:23.318361+0000","last_became_active":"2024-03-29T04:10:22.303477+0000","last_became_peered":"2024-03-29T04:10:22.303477+0000","last_unstale":"2024-03-29T04:10:23.318361+0000","last_undegraded":"2024-03-29T04:10:23.318361+0000","last_fullsized":"2024-03-29T04:10:23.318361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:03:07.972943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.310284+0000","last_change":"2024-03-29T04:10:23.306990+0000","last_active":"2024-03-29T04:10:23.310284+0000","last_peered":"2024-03-29T04:10:23.310284+0000","last_clean":"2024-03-29T04:10:23.310284+0000","last_became_active":"2024-03-29T04:10:23.306635+0000","last_became_peered":"2024-03-29T04:10:23.306635+0000","last_unstale":"2024-03-29T04:10:23.310284+0000","last_undegraded":"2024-03-29T04:10:23.310284+0000","last_fullsized":"2024-03-29T04:10:23.310284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:25.613005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297872+0000","last_change":"2024-03-29T04:10:22.300528+0000","last_active":"2024-03-29T04:10:23.297872+0000","last_peered":"2024-03-29T04:10:23.297872+0000","last_clean":"2024-03-29T04:10:23.297872+0000","last_became_active":"2024-03-29T04:10:22.299874+0000","last_became_peered":"2024-03-29T04:10:22.299874+0000","last_unstale":"2024-03-29T04:10:23.297872+0000","last_undegraded":"2024-03-29T04:10:23.297872+0000","last_fullsized":"2024-03-29T04:10:23.297872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316810+0000","last_change":"2024-03-29T04:10:23.311796+0000","last_active":"2024-03-29T04:10:23.316810+0000","last_peered":"2024-03-29T04:10:23.316810+0000","last_clean":"2024-03-29T04:10:23.316810+0000","last_became_active":"2024-03-29T04:10:23.311351+0000","last_became_peered":"2024-03-29T04:10:23.311351+0000","last_unstale":"2024-03-29T04:10:23.316810+0000","last_undegraded":"2024-03-29T04:10:23.316810+0000","last_fullsized":"2024-03-29T04:10:23.316810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309583+0000","last_change":"2024-03-29T04:10:22.308134+0000","last_active":"2024-03-29T04:10:23.309583+0000","last_peered":"2024-03-29T04:10:23.309583+0000","last_clean":"2024-03-29T04:10:23.309583+0000","last_became_active":"2024-03-29T04:10:22.307860+0000","last_became_peered":"2024-03-29T04:10:22.307860+0000","last_unstale":"2024-03-29T04:10:23.309583+0000","last_undegraded":"2024-03-29T04:10:23.309583+0000","last_fullsized":"2024-03-29T04:10:23.309583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:20:17.587318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311340+0000","last_change":"2024-03-29T04:10:23.309927+0000","last_active":"2024-03-29T04:10:23.311340+0000","last_peered":"2024-03-29T04:10:23.311340+0000","last_clean":"2024-03-29T04:10:23.311340+0000","last_became_active":"2024-03-29T04:10:23.309623+0000","last_became_peered":"2024-03-29T04:10:23.309623+0000","last_unstale":"2024-03-29T04:10:23.311340+0000","last_undegraded":"2024-03-29T04:10:23.311340+0000","last_fullsized":"2024-03-29T04:10:23.311340+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.301262+0000","last_change":"2024-03-29T04:10:22.297913+0000","last_active":"2024-03-29T04:10:23.301262+0000","last_peered":"2024-03-29T04:10:23.301262+0000","last_clean":"2024-03-29T04:10:23.301262+0000","last_became_active":"2024-03-29T04:10:22.296015+0000","last_became_peered":"2024-03-29T04:10:22.296015+0000","last_unstale":"2024-03-29T04:10:23.301262+0000","last_undegraded":"2024-03-29T04:10:23.301262+0000","last_fullsized":"2024-03-29T04:10:23.301262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:13.154992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300580+0000","last_change":"2024-03-29T04:10:22.300580+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300580+0000","last_undegraded":"2024-03-29T04:10:22.300580+0000","last_fullsized":"2024-03-29T04:10:22.300580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292907+0000","last_change":"2024-03-29T04:10:22.305287+0000","last_active":"2024-03-29T04:10:23.292907+0000","last_peered":"2024-03-29T04:10:23.292907+0000","last_clean":"2024-03-29T04:10:23.292907+0000","last_became_active":"2024-03-29T04:10:22.304999+0000","last_became_peered":"2024-03-29T04:10:22.304999+0000","last_unstale":"2024-03-29T04:10:23.292907+0000","last_undegraded":"2024-03-29T04:10:23.292907+0000","last_fullsized":"2024-03-29T04:10:23.292907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312347+0000","last_change":"2024-03-29T04:10:23.307638+0000","last_active":"2024-03-29T04:10:23.312347+0000","last_peered":"2024-03-29T04:10:23.312347+0000","last_clean":"2024-03-29T04:10:23.312347+0000","last_became_active":"2024-03-29T04:10:23.307381+0000","last_became_peered":"2024-03-29T04:10:23.307381+0000","last_unstale":"2024-03-29T04:10:23.312347+0000","last_undegraded":"2024-03-29T04:10:23.312347+0000","last_fullsized":"2024-03-29T04:10:23.312347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299841+0000","last_change":"2024-03-29T04:10:22.307254+0000","last_active":"2024-03-29T04:10:23.299841+0000","last_peered":"2024-03-29T04:10:23.299841+0000","last_clean":"2024-03-29T04:10:23.299841+0000","last_became_active":"2024-03-29T04:10:22.306976+0000","last_became_peered":"2024-03-29T04:10:22.306976+0000","last_unstale":"2024-03-29T04:10:23.299841+0000","last_undegraded":"2024-03-29T04:10:23.299841+0000","last_fullsized":"2024-03-29T04:10:23.299841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:54:11.970574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320128+0000","last_change":"2024-03-29T04:10:23.317609+0000","last_active":"2024-03-29T04:10:23.320128+0000","last_peered":"2024-03-29T04:10:23.320128+0000","last_clean":"2024-03-29T04:10:23.320128+0000","last_became_active":"2024-03-29T04:10:23.317364+0000","last_became_peered":"2024-03-29T04:10:23.317364+0000","last_unstale":"2024-03-29T04:10:23.320128+0000","last_undegraded":"2024-03-29T04:10:23.320128+0000","last_fullsized":"2024-03-29T04:10:23.320128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310663+0000","last_change":"2024-03-29T04:10:22.308920+0000","last_active":"2024-03-29T04:10:22.310663+0000","last_peered":"2024-03-29T04:10:22.310663+0000","last_clean":"2024-03-29T04:10:22.310663+0000","last_became_active":"2024-03-29T04:10:22.308648+0000","last_became_peered":"2024-03-29T04:10:22.308648+0000","last_unstale":"2024-03-29T04:10:22.310663+0000","last_undegraded":"2024-03-29T04:10:22.310663+0000","last_fullsized":"2024-03-29T04:10:22.310663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314664+0000","last_change":"2024-03-29T04:10:23.305533+0000","last_active":"2024-03-29T04:10:23.314664+0000","last_peered":"2024-03-29T04:10:23.314664+0000","last_clean":"2024-03-29T04:10:23.314664+0000","last_became_active":"2024-03-29T04:10:23.305295+0000","last_became_peered":"2024-03-29T04:10:23.305295+0000","last_unstale":"2024-03-29T04:10:23.314664+0000","last_undegraded":"2024-03-29T04:10:23.314664+0000","last_fullsized":"2024-03-29T04:10:23.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304196+0000","last_change":"2024-03-29T04:10:22.309267+0000","last_active":"2024-03-29T04:10:23.304196+0000","last_peered":"2024-03-29T04:10:23.304196+0000","last_clean":"2024-03-29T04:10:23.304196+0000","last_became_active":"2024-03-29T04:10:22.308996+0000","last_became_peered":"2024-03-29T04:10:22.308996+0000","last_unstale":"2024-03-29T04:10:23.304196+0000","last_undegraded":"2024-03-29T04:10:23.304196+0000","last_fullsized":"2024-03-29T04:10:23.304196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:27.354497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317732+0000","last_change":"2024-03-29T04:10:23.313551+0000","last_active":"2024-03-29T04:10:23.317732+0000","last_peered":"2024-03-29T04:10:23.317732+0000","last_clean":"2024-03-29T04:10:23.317732+0000","last_became_active":"2024-03-29T04:10:23.313289+0000","last_became_peered":"2024-03-29T04:10:23.313289+0000","last_unstale":"2024-03-29T04:10:23.317732+0000","last_undegraded":"2024-03-29T04:10:23.317732+0000","last_fullsized":"2024-03-29T04:10:23.317732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299143+0000","last_change":"2024-03-29T04:10:22.308648+0000","last_active":"2024-03-29T04:10:23.299143+0000","last_peered":"2024-03-29T04:10:23.299143+0000","last_clean":"2024-03-29T04:10:23.299143+0000","last_became_active":"2024-03-29T04:10:22.308334+0000","last_became_peered":"2024-03-29T04:10:22.308334+0000","last_unstale":"2024-03-29T04:10:23.299143+0000","last_undegraded":"2024-03-29T04:10:23.299143+0000","last_fullsized":"2024-03-29T04:10:23.299143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322380+0000","last_change":"2024-03-29T04:10:23.319502+0000","last_active":"2024-03-29T04:10:23.322380+0000","last_peered":"2024-03-29T04:10:23.322380+0000","last_clean":"2024-03-29T04:10:23.322380+0000","last_became_active":"2024-03-29T04:10:23.319209+0000","last_became_peered":"2024-03-29T04:10:23.319209+0000","last_unstale":"2024-03-29T04:10:23.322380+0000","last_undegraded":"2024-03-29T04:10:23.322380+0000","last_fullsized":"2024-03-29T04:10:23.322380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.306085+0000","last_change":"2024-03-29T04:10:22.307995+0000","last_active":"2024-03-29T04:10:23.306085+0000","last_peered":"2024-03-29T04:10:23.306085+0000","last_clean":"2024-03-29T04:10:23.306085+0000","last_became_active":"2024-03-29T04:10:22.307723+0000","last_became_peered":"2024-03-29T04:10:22.307723+0000","last_unstale":"2024-03-29T04:10:23.306085+0000","last_undegraded":"2024-03-29T04:10:23.306085+0000","last_fullsized":"2024-03-29T04:10:23.306085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316387+0000","last_change":"2024-03-29T04:10:23.308586+0000","last_active":"2024-03-29T04:10:23.316387+0000","last_peered":"2024-03-29T04:10:23.316387+0000","last_clean":"2024-03-29T04:10:23.316387+0000","last_became_active":"2024-03-29T04:10:23.308278+0000","last_became_peered":"2024-03-29T04:10:23.308278+0000","last_unstale":"2024-03-29T04:10:23.316387+0000","last_undegraded":"2024-03-29T04:10:23.316387+0000","last_fullsized":"2024-03-29T04:10:23.316387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295435+0000","last_change":"2024-03-29T04:10:22.309876+0000","last_active":"2024-03-29T04:10:23.295435+0000","last_peered":"2024-03-29T04:10:23.295435+0000","last_clean":"2024-03-29T04:10:23.295435+0000","last_became_active":"2024-03-29T04:10:22.309473+0000","last_became_peered":"2024-03-29T04:10:22.309473+0000","last_unstale":"2024-03-29T04:10:23.295435+0000","last_undegraded":"2024-03-29T04:10:23.295435+0000","last_fullsized":"2024-03-29T04:10:23.295435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:20.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292868+0000","last_change":"2024-03-29T04:10:22.292868+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292868+0000","last_undegraded":"2024-03-29T04:10:22.292868+0000","last_fullsized":"2024-03-29T04:10:22.292868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.303802+0000","last_change":"2024-03-29T04:10:22.302187+0000","last_active":"2024-03-29T04:10:22.303802+0000","last_peered":"2024-03-29T04:10:22.303802+0000","last_clean":"2024-03-29T04:10:22.303802+0000","last_became_active":"2024-03-29T04:10:22.301830+0000","last_became_peered":"2024-03-29T04:10:22.301830+0000","last_unstale":"2024-03-29T04:10:22.303802+0000","last_undegraded":"2024-03-29T04:10:22.303802+0000","last_fullsized":"2024-03-29T04:10:22.303802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314171+0000","last_change":"2024-03-29T04:10:23.303127+0000","last_active":"2024-03-29T04:10:23.314171+0000","last_peered":"2024-03-29T04:10:23.314171+0000","last_clean":"2024-03-29T04:10:23.314171+0000","last_became_active":"2024-03-29T04:10:23.302882+0000","last_became_peered":"2024-03-29T04:10:23.302882+0000","last_unstale":"2024-03-29T04:10:23.314171+0000","last_undegraded":"2024-03-29T04:10:23.314171+0000","last_fullsized":"2024-03-29T04:10:23.314171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297157+0000","last_change":"2024-03-29T04:10:22.304632+0000","last_active":"2024-03-29T04:10:23.297157+0000","last_peered":"2024-03-29T04:10:23.297157+0000","last_clean":"2024-03-29T04:10:23.297157+0000","last_became_active":"2024-03-29T04:10:22.304363+0000","last_became_peered":"2024-03-29T04:10:22.304363+0000","last_unstale":"2024-03-29T04:10:23.297157+0000","last_undegraded":"2024-03-29T04:10:23.297157+0000","last_fullsized":"2024-03-29T04:10:23.297157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:35.091291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301182+0000","last_change":"2024-03-29T04:10:22.301182+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301182+0000","last_undegraded":"2024-03-29T04:10:22.301182+0000","last_fullsized":"2024-03-29T04:10:22.301182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.293779+0000","last_change":"2024-03-29T04:10:22.310649+0000","last_active":"2024-03-29T04:10:23.293779+0000","last_peered":"2024-03-29T04:10:23.293779+0000","last_clean":"2024-03-29T04:10:23.293779+0000","last_became_active":"2024-03-29T04:10:22.310382+0000","last_became_peered":"2024-03-29T04:10:22.310382+0000","last_unstale":"2024-03-29T04:10:23.293779+0000","last_undegraded":"2024-03-29T04:10:23.293779+0000","last_fullsized":"2024-03-29T04:10:23.293779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292459+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:23.292459+0000","last_peered":"2024-03-29T04:10:23.292459+0000","last_clean":"2024-03-29T04:10:23.292459+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:23.292459+0000","last_undegraded":"2024-03-29T04:10:23.292459+0000","last_fullsized":"2024-03-29T04:10:23.292459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":192127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.302395+0000', 'last_change': '2024-03-29T04:10:22.302395+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.302395+0000', 'last_undegraded': '2024-03-29T04:10:22.302395+0000', 'last_fullsized': '2024-03-29T04:10:22.302395+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.303882+0000', 'last_change': '2024-03-29T04:10:22.303882+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.303882+0000', 'last_undegraded': '2024-03-29T04:10:22.303882+0000', 'last_fullsized': '2024-03-29T04:10:22.303882+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-29T04:10:27.203 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.301429+0000', 'last_change': '2024-03-29T04:10:22.301429+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.301429+0000', 'last_undegraded': '2024-03-29T04:10:22.301429+0000', 'last_fullsized': '2024-03-29T04:10:22.301429+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.300951+0000', 'last_change': '2024-03-29T04:10:22.300951+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.300951+0000', 'last_undegraded': '2024-03-29T04:10:22.300951+0000', 'last_fullsized': '2024-03-29T04:10:22.300951+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.293775+0000', 'last_change': '2024-03-29T04:10:22.293775+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.293775+0000', 'last_undegraded': '2024-03-29T04:10:22.293775+0000', 'last_fullsized': '2024-03-29T04:10:22.293775+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.301942+0000', 'last_change': '2024-03-29T04:10:22.301942+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.301942+0000', 'last_undegraded': '2024-03-29T04:10:22.301942+0000', 'last_fullsized': '2024-03-29T04:10:22.301942+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.300543+0000', 'last_change': '2024-03-29T04:10:22.300543+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.300543+0000', 'last_undegraded': '2024-03-29T04:10:22.300543+0000', 'last_fullsized': '2024-03-29T04:10:22.300543+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.291376+0000', 'last_change': '2024-03-29T04:10:22.291376+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.291376+0000', 'last_undegraded': '2024-03-29T04:10:22.291376+0000', 'last_fullsized': '2024-03-29T04:10:22.291376+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.204 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.299555+0000', 'last_change': '2024-03-29T04:10:22.299555+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.299555+0000', 'last_undegraded': '2024-03-29T04:10:22.299555+0000', 'last_fullsized': '2024-03-29T04:10:22.299555+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.291935+0000', 'last_change': '2024-03-29T04:10:22.291935+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.291935+0000', 'last_undegraded': '2024-03-29T04:10:22.291935+0000', 'last_fullsized': '2024-03-29T04:10:22.291935+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.301511+0000', 'last_change': '2024-03-29T04:10:22.301511+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.301511+0000', 'last_undegraded': '2024-03-29T04:10:22.301511+0000', 'last_fullsized': '2024-03-29T04:10:22.301511+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.292420+0000', 'last_change': '2024-03-29T04:10:22.292420+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.292420+0000', 'last_undegraded': '2024-03-29T04:10:22.292420+0000', 'last_fullsized': '2024-03-29T04:10:22.292420+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.292102+0000', 'last_change': '2024-03-29T04:10:22.292102+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.292102+0000', 'last_undegraded': '2024-03-29T04:10:22.292102+0000', 'last_fullsized': '2024-03-29T04:10:22.292102+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-29T04:10:27.205 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.301030+0000', 'last_change': '2024-03-29T04:10:22.301030+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.301030+0000', 'last_undegraded': '2024-03-29T04:10:22.301030+0000', 'last_fullsized': '2024-03-29T04:10:22.301030+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.292383+0000', 'last_change': '2024-03-29T04:10:22.292383+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.292383+0000', 'last_undegraded': '2024-03-29T04:10:22.292383+0000', 'last_fullsized': '2024-03-29T04:10:22.292383+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.300580+0000', 'last_change': '2024-03-29T04:10:22.300580+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.300580+0000', 'last_undegraded': '2024-03-29T04:10:22.300580+0000', 'last_fullsized': '2024-03-29T04:10:22.300580+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.292868+0000', 'last_change': '2024-03-29T04:10:22.292868+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.292868+0000', 'last_undegraded': '2024-03-29T04:10:22.292868+0000', 'last_fullsized': '2024-03-29T04:10:22.292868+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-29T04:10:27.206 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-03-29T04:10:22.301182+0000', 'last_change': '2024-03-29T04:10:22.301182+0000', 'last_active': '2024-03-29T04:10:22.281232+0000', 'last_peered': '2024-03-29T04:10:22.281232+0000', 'last_clean': '2024-03-29T04:10:22.281232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T04:10:22.301182+0000', 'last_undegraded': '2024-03-29T04:10:22.301182+0000', 'last_fullsized': '2024-03-29T04:10:22.301182+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'last_clean_scrub_stamp': '2024-03-29T04:10:22.281232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T04:10:27.207 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T04:10:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.362+0000 7f9053959640 1 Processor -- start 2024-03-29T04:10:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 -- start start 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f904c0d4a90 0x7f904c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c0d2f00 0x7f904c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c0d36f0 con 0x7f904c15e950 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f904c0d3cb0 con 0x7f904c0d4a90 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9053959640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0d39d0 con 0x7f904c0d2f00 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f9050ef4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f904c0d4a90 0x7f904c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f90516f5640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f904c15e950 0x7f904c0d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54584/0 (socket says 172.21.15.181:54584) 2024-03-29T04:10:27.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.363+0000 7f90516f5640 1 -- 172.21.15.181:0/1543496848 learned_addr learned my addr 172.21.15.181:0/1543496848 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9051ef6640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c0d2f00 0x7f904c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9050ef4640 1 -- 172.21.15.181:0/1543496848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c0d2f00 msgr2=0x7f904c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9050ef4640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c0d2f00 0x7f904c0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9050ef4640 1 -- 172.21.15.181:0/1543496848 >> v1:172.21.15.181:6789/0 conn(0x7f904c15e950 legacy=0x7f904c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9050ef4640 1 -- 172.21.15.181:0/1543496848 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0b7ac0 con 0x7f904c0d4a90 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9043fff640 1 -- 172.21.15.181:0/1543496848 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930603006 0 0) 0x7f904c0d36f0 con 0x7f904c15e950 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.364+0000 7f9050ef4640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d4a90 0x7f904c0d29c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097a0 tx=0x7f903c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f33c487c2892b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9043fff640 1 -- 172.21.15.181:0/1543496848 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c034070 con 0x7f904c0d4a90 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9043fff640 1 -- 172.21.15.181:0/1543496848 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903c030610 con 0x7f904c0d4a90 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9043fff640 1 -- 172.21.15.181:0/1543496848 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0309e0 con 0x7f904c0d4a90 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 -- 172.21.15.181:0/1543496848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d4a90 msgr2=0x7f904c0d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d4a90 0x7f904c0d29c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f903c0097a0 tx=0x7f903c02b960 comp rx=0 tx=0).stop 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 -- 172.21.15.181:0/1543496848 shutdown_connections 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c0d2f00 0x7f904c0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.346 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 --2- 172.21.15.181:0/1543496848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d4a90 0x7f904c0d29c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 -- 172.21.15.181:0/1543496848 >> 172.21.15.181:0/1543496848 conn(0x7f904c0676e0 msgr2=0x7f904c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 -- 172.21.15.181:0/1543496848 shutdown_connections 2024-03-29T04:10:27.347 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.365+0000 7f9053959640 1 -- 172.21.15.181:0/1543496848 wait complete. 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.366+0000 7f9053959640 1 Processor -- start 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.366+0000 7f9053959640 1 -- start start 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c0d4a90 0x7f904c0ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c15e950 0x7f904c0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f904c15ed30 con 0x7f904c0d2f00 2024-03-29T04:10:27.349 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0d33b0 con 0x7f904c0d4a90 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f9053959640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f904c15eeb0 con 0x7f904c15e950 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55164/0 (socket says 172.21.15.181:55164) 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 -- 172.21.15.181:0/3149816166 learned_addr learned my addr 172.21.15.181:0/3149816166 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 -- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c15e950 msgr2=0x7f904c0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c15e950 0x7f904c0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 -- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c0d4a90 msgr2=0x7f904c0ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c0d4a90 0x7f904c0ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.367+0000 7f90516f5640 1 -- 172.21.15.181:0/3149816166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0af420 con 0x7f904c0d2f00 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f90516f5640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f904800c500 tx=0x7f904800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048018070 con 0x7f904c0d2f00 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9048007560 con 0x7f904c0d2f00 2024-03-29T04:10:27.350 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c0ade40 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004a20 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.368+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c0ae400 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.369+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c075a10 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.369+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f904801b070 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.370+0000 7f9041ffb640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 0x7f903006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.370+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f904808ab40 con 0x7f904c0d2f00 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.370+0000 7f9050ef4640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 0x7f903006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.370+0000 7f9050ef4640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 0x7f903006ea20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f903c036000 tx=0x7f903c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.374+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9048058fd0 con 0x7f904c0d2f00 2024-03-29T04:10:27.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.627+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f904c0cf4f0 con 0x7f904c0d2f00 2024-03-29T04:10:27.607 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.628+0000 7f9041ffb640 1 -- 172.21.15.181:0/3149816166 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1259 (secure 0 0 0) 0x7f90480588d0 con 0x7f904c0d2f00 2024-03-29T04:10:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:27.607 INFO:teuthology.orchestra.run.smithi181.stdout:{"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":96,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1711685342,"num_in_osds":8,"osd_in_since":1711685333,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":79},{"state_name":"creating+peering","count":18}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592666,"bytes_used":225980416,"bytes_avail":772868132864,"bytes_total":773094113280,"inactive_pgs_ratio":0.18556700646877289,"write_bytes_sec":2853,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-29T04:10:11.584948+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-29T04:10:27.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 msgr2=0x7f903006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 0x7f903006ea20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f903c036000 tx=0x7f903c030000 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 msgr2=0x7f904c0ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f904800c500 tx=0x7f904800c9c0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 shutdown_connections 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f903006c560 0x7f903006ea20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f904c15e950 0x7f904c0aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f904c0d4a90 0x7f904c0ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 --2- 172.21.15.181:0/3149816166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c0d2f00 0x7f904c0ad760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 >> 172.21.15.181:0/3149816166 conn(0x7f904c0676e0 msgr2=0x7f904c0cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 shutdown_connections 2024-03-29T04:10:27.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.630+0000 7f9053959640 1 -- 172.21.15.181:0/3149816166 wait complete. 2024-03-29T04:10:27.620 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T04:10:27.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.742+0000 7f1ea2715640 1 Processor -- start 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.742+0000 7f1ea2715640 1 -- start start 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea2715640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e9c072230 0x7f1e9c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea2715640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c06db10 0x7f1e9c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea2715640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e9c06c4d0 con 0x7f1e9c072610 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea2715640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9c158f10 con 0x7f1e9c072230 2024-03-29T04:10:27.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea2715640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9c15ef70 con 0x7f1e9c06db10 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e9c072230 0x7f1e9c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9b7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1e9c072610 0x7f1e9c15f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39088/0 (socket says 172.21.15.181:39088) 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9b7fe640 1 -- 172.21.15.181:0/636949345 learned_addr learned my addr 172.21.15.181:0/636949345 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1ea0cb2640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c06db10 0x7f1e9c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9bfff640 1 -- 172.21.15.181:0/636949345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c06db10 msgr2=0x7f1e9c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9bfff640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c06db10 0x7f1e9c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9bfff640 1 -- 172.21.15.181:0/636949345 >> v1:172.21.15.181:6789/0 conn(0x7f1e9c072610 legacy=0x7f1e9c15f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9bfff640 1 -- 172.21.15.181:0/636949345 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9c0ce840 con 0x7f1e9c072230 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.743+0000 7f1e9affd640 1 -- 172.21.15.181:0/636949345 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395878761 0 0) 0x7f1e84001110 con 0x7f1e9c072610 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.744+0000 7f1e9bfff640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c072230 0x7f1e9c06bf90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33038ed2911db7f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.744+0000 7f1e9affd640 1 -- 172.21.15.181:0/636949345 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c035020 con 0x7f1e9c072230 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.744+0000 7f1e9affd640 1 -- 172.21.15.181:0/636949345 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c031830 con 0x7f1e9c072230 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.744+0000 7f1e9affd640 1 -- 172.21.15.181:0/636949345 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c031c00 con 0x7f1e9c072230 2024-03-29T04:10:27.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 -- 172.21.15.181:0/636949345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c072230 msgr2=0x7f1e9c06bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c072230 0x7f1e9c06bf90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02b8d0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 -- 172.21.15.181:0/636949345 shutdown_connections 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c06db10 0x7f1e9c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 --2- 172.21.15.181:0/636949345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c072230 0x7f1e9c06bf90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 -- 172.21.15.181:0/636949345 >> 172.21.15.181:0/636949345 conn(0x7f1e9c0d0cb0 msgr2=0x7f1e9c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 -- 172.21.15.181:0/636949345 shutdown_connections 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.745+0000 7f1ea2715640 1 -- 172.21.15.181:0/636949345 wait complete. 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.746+0000 7f1ea2715640 1 Processor -- start 2024-03-29T04:10:27.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.746+0000 7f1ea2715640 1 -- start start 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c072230 0x7f1e9c143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 0x7f1e9c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9c072ac0 con 0x7f1e9c06db10 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9c15ec00 con 0x7f1e9c072610 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9c06fa90 con 0x7f1e9c072230 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39566/0 (socket says 172.21.15.181:39566) 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 -- 172.21.15.181:0/2781066616 learned_addr learned my addr 172.21.15.181:0/2781066616 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9b7fe640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c072230 0x7f1e9c143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea0cb2640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 0x7f1e9c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 -- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c072230 msgr2=0x7f1e9c143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c072230 0x7f1e9c143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 -- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 msgr2=0x7f1e9c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 0x7f1e9c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 -- 172.21.15.181:0/2781066616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9c1604c0 con 0x7f1e9c06db10 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea0cb2640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 0x7f1e9c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e9bfff640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000c00 tx=0x7f1e8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c035070 con 0x7f1e9c06db10 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c004230 con 0x7f1e9c06db10 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c03a630 con 0x7f1e9c06db10 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.748+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9c1607a0 con 0x7f1e9c06db10 2024-03-29T04:10:27.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.749+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e9c161040 con 0x7f1e9c06db10 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.749+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e9c160b90 con 0x7f1e9c06db10 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.750+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1e8c038020 con 0x7f1e9c06db10 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.750+0000 7f1e98ff9640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 0x7f1e7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.751+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1e8c0ad570 con 0x7f1e9c06db10 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.751+0000 7f1e9b7fe640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 0x7f1e7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:27.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.751+0000 7f1e9b7fe640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 0x7f1e7c06e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c1628f0 tx=0x7f1e84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:27.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.754+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1e8c040070 con 0x7f1e9c06db10 2024-03-29T04:10:27.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.932+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e9c075a10 con 0x7f1e7c06c530 2024-03-29T04:10:27.917 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.938+0000 7f1e98ff9640 1 -- 172.21.15.181:0/2781066616 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250415 (secure 0 0 0) 0x7f1e9c075a10 con 0x7f1e7c06c530 2024-03-29T04:10:27.920 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:27.921 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:10:27.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 msgr2=0x7f1e7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 0x7f1e7c06e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c1628f0 tx=0x7f1e84009040 comp rx=0 tx=0).stop 2024-03-29T04:10:27.922 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 msgr2=0x7f1e9c15fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000c00 tx=0x7f1e8c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 shutdown_connections 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e9c072610 0x7f1e9c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e7c06c530 0x7f1e7c06e9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e9c072230 0x7f1e9c143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 --2- 172.21.15.181:0/2781066616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c06db10 0x7f1e9c15fee0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 >> 172.21.15.181:0/2781066616 conn(0x7f1e9c0d0cb0 msgr2=0x7f1e9c15c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 shutdown_connections 2024-03-29T04:10:27.923 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:27.941+0000 7f1ea2715640 1 -- 172.21.15.181:0/2781066616 wait complete. 2024-03-29T04:10:27.931 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-03-29T04:10:27.587326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220684,"kb_used_data":4508,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754036,"statfs":{"total":773094113280,"available":772868132864,"internally_reserved":0,"allocated":4616192,"data_stored":2649928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000891"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318073+0000","last_change":"2024-03-29T04:10:23.313049+0000","last_active":"2024-03-29T04:10:23.318073+0000","last_peered":"2024-03-29T04:10:23.318073+0000","last_clean":"2024-03-29T04:10:23.318073+0000","last_became_active":"2024-03-29T04:10:23.312619+0000","last_became_peered":"2024-03-29T04:10:23.312619+0000","last_unstale":"2024-03-29T04:10:23.318073+0000","last_undegraded":"2024-03-29T04:10:23.318073+0000","last_fullsized":"2024-03-29T04:10:23.318073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309796+0000","last_change":"2024-03-29T04:10:23.304958+0000","last_active":"2024-03-29T04:10:23.309796+0000","last_peered":"2024-03-29T04:10:23.309796+0000","last_clean":"2024-03-29T04:10:23.309796+0000","last_became_active":"2024-03-29T04:10:23.304714+0000","last_became_peered":"2024-03-29T04:10:23.304714+0000","last_unstale":"2024-03-29T04:10:23.309796+0000","last_undegraded":"2024-03-29T04:10:23.309796+0000","last_fullsized":"2024-03-29T04:10:23.309796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322299+0000","last_change":"2024-03-29T04:10:23.315527+0000","last_active":"2024-03-29T04:10:23.322299+0000","last_peered":"2024-03-29T04:10:23.322299+0000","last_clean":"2024-03-29T04:10:23.322299+0000","last_became_active":"2024-03-29T04:10:23.315242+0000","last_became_peered":"2024-03-29T04:10:23.315242+0000","last_unstale":"2024-03-29T04:10:23.322299+0000","last_undegraded":"2024-03-29T04:10:23.322299+0000","last_fullsized":"2024-03-29T04:10:23.322299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321235+0000","last_change":"2024-03-29T04:10:23.315865+0000","last_active":"2024-03-29T04:10:23.321235+0000","last_peered":"2024-03-29T04:10:23.321235+0000","last_clean":"2024-03-29T04:10:23.321235+0000","last_became_active":"2024-03-29T04:10:23.315580+0000","last_became_peered":"2024-03-29T04:10:23.315580+0000","last_unstale":"2024-03-29T04:10:23.321235+0000","last_undegraded":"2024-03-29T04:10:23.321235+0000","last_fullsized":"2024-03-29T04:10:23.321235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:33:01.952778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321146+0000","last_change":"2024-03-29T04:10:23.315150+0000","last_active":"2024-03-29T04:10:23.321146+0000","last_peered":"2024-03-29T04:10:23.321146+0000","last_clean":"2024-03-29T04:10:23.321146+0000","last_became_active":"2024-03-29T04:10:23.314879+0000","last_became_peered":"2024-03-29T04:10:23.314879+0000","last_unstale":"2024-03-29T04:10:23.321146+0000","last_undegraded":"2024-03-29T04:10:23.321146+0000","last_fullsized":"2024-03-29T04:10:23.321146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.011645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316771+0000","last_change":"2024-03-29T04:10:23.311220+0000","last_active":"2024-03-29T04:10:23.316771+0000","last_peered":"2024-03-29T04:10:23.316771+0000","last_clean":"2024-03-29T04:10:23.316771+0000","last_became_active":"2024-03-29T04:10:23.310727+0000","last_became_peered":"2024-03-29T04:10:23.310727+0000","last_unstale":"2024-03-29T04:10:23.316771+0000","last_undegraded":"2024-03-29T04:10:23.316771+0000","last_fullsized":"2024-03-29T04:10:23.316771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.302395+0000","last_change":"2024-03-29T04:10:22.302395+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.302395+0000","last_undegraded":"2024-03-29T04:10:22.302395+0000","last_fullsized":"2024-03-29T04:10:22.302395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317690+0000","last_change":"2024-03-29T04:10:23.314124+0000","last_active":"2024-03-29T04:10:23.317690+0000","last_peered":"2024-03-29T04:10:23.317690+0000","last_clean":"2024-03-29T04:10:23.317690+0000","last_became_active":"2024-03-29T04:10:23.313434+0000","last_became_peered":"2024-03-29T04:10:23.313434+0000","last_unstale":"2024-03-29T04:10:23.317690+0000","last_undegraded":"2024-03-29T04:10:23.317690+0000","last_fullsized":"2024-03-29T04:10:23.317690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322272+0000","last_change":"2024-03-29T04:10:23.315461+0000","last_active":"2024-03-29T04:10:23.322272+0000","last_peered":"2024-03-29T04:10:23.322272+0000","last_clean":"2024-03-29T04:10:23.322272+0000","last_became_active":"2024-03-29T04:10:23.315163+0000","last_became_peered":"2024-03-29T04:10:23.315163+0000","last_unstale":"2024-03-29T04:10:23.322272+0000","last_undegraded":"2024-03-29T04:10:23.322272+0000","last_fullsized":"2024-03-29T04:10:23.322272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.303882+0000","last_change":"2024-03-29T04:10:22.303882+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.303882+0000","last_undegraded":"2024-03-29T04:10:22.303882+0000","last_fullsized":"2024-03-29T04:10:22.303882+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301429+0000","last_change":"2024-03-29T04:10:22.301429+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301429+0000","last_undegraded":"2024-03-29T04:10:22.301429+0000","last_fullsized":"2024-03-29T04:10:22.301429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311095+0000","last_change":"2024-03-29T04:10:23.309450+0000","last_active":"2024-03-29T04:10:23.311095+0000","last_peered":"2024-03-29T04:10:23.311095+0000","last_clean":"2024-03-29T04:10:23.311095+0000","last_became_active":"2024-03-29T04:10:23.309186+0000","last_became_peered":"2024-03-29T04:10:23.309186+0000","last_unstale":"2024-03-29T04:10:23.311095+0000","last_undegraded":"2024-03-29T04:10:23.311095+0000","last_fullsized":"2024-03-29T04:10:23.311095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.135609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300951+0000","last_change":"2024-03-29T04:10:22.300951+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300951+0000","last_undegraded":"2024-03-29T04:10:22.300951+0000","last_fullsized":"2024-03-29T04:10:22.300951+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.293775+0000","last_change":"2024-03-29T04:10:22.293775+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.293775+0000","last_undegraded":"2024-03-29T04:10:22.293775+0000","last_fullsized":"2024-03-29T04:10:22.293775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301942+0000","last_change":"2024-03-29T04:10:22.301942+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301942+0000","last_undegraded":"2024-03-29T04:10:22.301942+0000","last_fullsized":"2024-03-29T04:10:22.301942+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316831+0000","last_change":"2024-03-29T04:10:23.311958+0000","last_active":"2024-03-29T04:10:23.316831+0000","last_peered":"2024-03-29T04:10:23.316831+0000","last_clean":"2024-03-29T04:10:23.316831+0000","last_became_active":"2024-03-29T04:10:23.311700+0000","last_became_peered":"2024-03-29T04:10:23.311700+0000","last_unstale":"2024-03-29T04:10:23.316831+0000","last_undegraded":"2024-03-29T04:10:23.316831+0000","last_fullsized":"2024-03-29T04:10:23.316831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316930+0000","last_change":"2024-03-29T04:10:23.313386+0000","last_active":"2024-03-29T04:10:23.316930+0000","last_peered":"2024-03-29T04:10:23.316930+0000","last_clean":"2024-03-29T04:10:23.316930+0000","last_became_active":"2024-03-29T04:10:23.313083+0000","last_became_peered":"2024-03-29T04:10:23.313083+0000","last_unstale":"2024-03-29T04:10:23.316930+0000","last_undegraded":"2024-03-29T04:10:23.316930+0000","last_fullsized":"2024-03-29T04:10:23.316930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304763+0000","last_change":"2024-03-29T04:10:23.300122+0000","last_active":"2024-03-29T04:10:23.304763+0000","last_peered":"2024-03-29T04:10:23.304763+0000","last_clean":"2024-03-29T04:10:23.304763+0000","last_became_active":"2024-03-29T04:10:23.299864+0000","last_became_peered":"2024-03-29T04:10:23.299864+0000","last_unstale":"2024-03-29T04:10:23.304763+0000","last_undegraded":"2024-03-29T04:10:23.304763+0000","last_fullsized":"2024-03-29T04:10:23.304763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:29.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321486+0000","last_change":"2024-03-29T04:10:23.316162+0000","last_active":"2024-03-29T04:10:23.321486+0000","last_peered":"2024-03-29T04:10:23.321486+0000","last_clean":"2024-03-29T04:10:23.321486+0000","last_became_active":"2024-03-29T04:10:23.315894+0000","last_became_peered":"2024-03-29T04:10:23.315894+0000","last_unstale":"2024-03-29T04:10:23.321486+0000","last_undegraded":"2024-03-29T04:10:23.321486+0000","last_fullsized":"2024-03-29T04:10:23.321486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:56:37.304758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.305943+0000","last_change":"2024-03-29T04:10:23.297888+0000","last_active":"2024-03-29T04:10:23.305943+0000","last_peered":"2024-03-29T04:10:23.305943+0000","last_clean":"2024-03-29T04:10:23.305943+0000","last_became_active":"2024-03-29T04:10:23.297611+0000","last_became_peered":"2024-03-29T04:10:23.297611+0000","last_unstale":"2024-03-29T04:10:23.305943+0000","last_undegraded":"2024-03-29T04:10:23.305943+0000","last_fullsized":"2024-03-29T04:10:23.305943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:51:35.316362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300543+0000","last_change":"2024-03-29T04:10:22.300543+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300543+0000","last_undegraded":"2024-03-29T04:10:22.300543+0000","last_fullsized":"2024-03-29T04:10:22.300543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291376+0000","last_change":"2024-03-29T04:10:22.291376+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291376+0000","last_undegraded":"2024-03-29T04:10:22.291376+0000","last_fullsized":"2024-03-29T04:10:22.291376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317436+0000","last_change":"2024-03-29T04:10:23.315012+0000","last_active":"2024-03-29T04:10:23.317436+0000","last_peered":"2024-03-29T04:10:23.317436+0000","last_clean":"2024-03-29T04:10:23.317436+0000","last_became_active":"2024-03-29T04:10:23.312553+0000","last_became_peered":"2024-03-29T04:10:23.312553+0000","last_unstale":"2024-03-29T04:10:23.317436+0000","last_undegraded":"2024-03-29T04:10:23.317436+0000","last_fullsized":"2024-03-29T04:10:23.317436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:36.093598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322166+0000","last_change":"2024-03-29T04:10:23.318761+0000","last_active":"2024-03-29T04:10:23.322166+0000","last_peered":"2024-03-29T04:10:23.322166+0000","last_clean":"2024-03-29T04:10:23.322166+0000","last_became_active":"2024-03-29T04:10:23.318414+0000","last_became_peered":"2024-03-29T04:10:23.318414+0000","last_unstale":"2024-03-29T04:10:23.322166+0000","last_undegraded":"2024-03-29T04:10:23.322166+0000","last_fullsized":"2024-03-29T04:10:23.322166+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:40.265277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319387+0000","last_change":"2024-03-29T04:10:23.312068+0000","last_active":"2024-03-29T04:10:23.319387+0000","last_peered":"2024-03-29T04:10:23.319387+0000","last_clean":"2024-03-29T04:10:23.319387+0000","last_became_active":"2024-03-29T04:10:23.311776+0000","last_became_peered":"2024-03-29T04:10:23.311776+0000","last_unstale":"2024-03-29T04:10:23.319387+0000","last_undegraded":"2024-03-29T04:10:23.319387+0000","last_fullsized":"2024-03-29T04:10:23.319387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:59.921229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320160+0000","last_change":"2024-03-29T04:10:23.318246+0000","last_active":"2024-03-29T04:10:23.320160+0000","last_peered":"2024-03-29T04:10:23.320160+0000","last_clean":"2024-03-29T04:10:23.320160+0000","last_became_active":"2024-03-29T04:10:23.317989+0000","last_became_peered":"2024-03-29T04:10:23.317989+0000","last_unstale":"2024-03-29T04:10:23.320160+0000","last_undegraded":"2024-03-29T04:10:23.320160+0000","last_fullsized":"2024-03-29T04:10:23.320160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309790+0000","last_change":"2024-03-29T04:10:23.308341+0000","last_active":"2024-03-29T04:10:23.309790+0000","last_peered":"2024-03-29T04:10:23.309790+0000","last_clean":"2024-03-29T04:10:23.309790+0000","last_became_active":"2024-03-29T04:10:23.307789+0000","last_became_peered":"2024-03-29T04:10:23.307789+0000","last_unstale":"2024-03-29T04:10:23.309790+0000","last_undegraded":"2024-03-29T04:10:23.309790+0000","last_fullsized":"2024-03-29T04:10:23.309790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:08.599452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.299555+0000","last_change":"2024-03-29T04:10:22.299555+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.299555+0000","last_undegraded":"2024-03-29T04:10:22.299555+0000","last_fullsized":"2024-03-29T04:10:22.299555+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308971+0000","last_change":"2024-03-29T04:10:23.307876+0000","last_active":"2024-03-29T04:10:23.308971+0000","last_peered":"2024-03-29T04:10:23.308971+0000","last_clean":"2024-03-29T04:10:23.308971+0000","last_became_active":"2024-03-29T04:10:23.307565+0000","last_became_peered":"2024-03-29T04:10:23.307565+0000","last_unstale":"2024-03-29T04:10:23.308971+0000","last_undegraded":"2024-03-29T04:10:23.308971+0000","last_fullsized":"2024-03-29T04:10:23.308971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:11.709686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.308760+0000","last_change":"2024-03-29T04:10:23.306970+0000","last_active":"2024-03-29T04:10:23.308760+0000","last_peered":"2024-03-29T04:10:23.308760+0000","last_clean":"2024-03-29T04:10:23.308760+0000","last_became_active":"2024-03-29T04:10:23.306708+0000","last_became_peered":"2024-03-29T04:10:23.306708+0000","last_unstale":"2024-03-29T04:10:23.308760+0000","last_undegraded":"2024-03-29T04:10:23.308760+0000","last_fullsized":"2024-03-29T04:10:23.308760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:15.611882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317417+0000","last_change":"2024-03-29T04:10:23.314775+0000","last_active":"2024-03-29T04:10:23.317417+0000","last_peered":"2024-03-29T04:10:23.317417+0000","last_clean":"2024-03-29T04:10:23.317417+0000","last_became_active":"2024-03-29T04:10:23.312160+0000","last_became_peered":"2024-03-29T04:10:23.312160+0000","last_unstale":"2024-03-29T04:10:23.317417+0000","last_undegraded":"2024-03-29T04:10:23.317417+0000","last_fullsized":"2024-03-29T04:10:23.317417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:01.861829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.313388+0000","last_change":"2024-03-29T04:10:23.304939+0000","last_active":"2024-03-29T04:10:23.313388+0000","last_peered":"2024-03-29T04:10:23.313388+0000","last_clean":"2024-03-29T04:10:23.313388+0000","last_became_active":"2024-03-29T04:10:23.304682+0000","last_became_peered":"2024-03-29T04:10:23.304682+0000","last_unstale":"2024-03-29T04:10:23.313388+0000","last_undegraded":"2024-03-29T04:10:23.313388+0000","last_fullsized":"2024-03-29T04:10:23.313388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323623+0000","last_change":"2024-03-29T04:10:23.318116+0000","last_active":"2024-03-29T04:10:23.323623+0000","last_peered":"2024-03-29T04:10:23.323623+0000","last_clean":"2024-03-29T04:10:23.323623+0000","last_became_active":"2024-03-29T04:10:23.317859+0000","last_became_peered":"2024-03-29T04:10:23.317859+0000","last_unstale":"2024-03-29T04:10:23.323623+0000","last_undegraded":"2024-03-29T04:10:23.323623+0000","last_fullsized":"2024-03-29T04:10:23.323623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319317+0000","last_change":"2024-03-29T04:10:22.297429+0000","last_active":"2024-03-29T04:10:23.319317+0000","last_peered":"2024-03-29T04:10:23.319317+0000","last_clean":"2024-03-29T04:10:23.319317+0000","last_became_active":"2024-03-29T04:10:22.297077+0000","last_became_peered":"2024-03-29T04:10:22.297077+0000","last_unstale":"2024-03-29T04:10:23.319317+0000","last_undegraded":"2024-03-29T04:10:23.319317+0000","last_fullsized":"2024-03-29T04:10:23.319317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:10.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320194+0000","last_change":"2024-03-29T04:10:23.316949+0000","last_active":"2024-03-29T04:10:23.320194+0000","last_peered":"2024-03-29T04:10:23.320194+0000","last_clean":"2024-03-29T04:10:23.320194+0000","last_became_active":"2024-03-29T04:10:23.316665+0000","last_became_peered":"2024-03-29T04:10:23.316665+0000","last_unstale":"2024-03-29T04:10:23.320194+0000","last_undegraded":"2024-03-29T04:10:23.320194+0000","last_fullsized":"2024-03-29T04:10:23.320194+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316490+0000","last_change":"2024-03-29T04:10:22.307974+0000","last_active":"2024-03-29T04:10:23.316490+0000","last_peered":"2024-03-29T04:10:23.316490+0000","last_clean":"2024-03-29T04:10:23.316490+0000","last_became_active":"2024-03-29T04:10:22.306547+0000","last_became_peered":"2024-03-29T04:10:22.306547+0000","last_unstale":"2024-03-29T04:10:23.316490+0000","last_undegraded":"2024-03-29T04:10:23.316490+0000","last_fullsized":"2024-03-29T04:10:23.316490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.291935+0000","last_change":"2024-03-29T04:10:22.291935+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.291935+0000","last_undegraded":"2024-03-29T04:10:22.291935+0000","last_fullsized":"2024-03-29T04:10:22.291935+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319454+0000","last_change":"2024-03-29T04:10:22.301842+0000","last_active":"2024-03-29T04:10:23.319454+0000","last_peered":"2024-03-29T04:10:23.319454+0000","last_clean":"2024-03-29T04:10:23.319454+0000","last_became_active":"2024-03-29T04:10:22.301559+0000","last_became_peered":"2024-03-29T04:10:22.301559+0000","last_unstale":"2024-03-29T04:10:23.319454+0000","last_undegraded":"2024-03-29T04:10:23.319454+0000","last_fullsized":"2024-03-29T04:10:23.319454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311930+0000","last_change":"2024-03-29T04:10:23.308697+0000","last_active":"2024-03-29T04:10:23.311930+0000","last_peered":"2024-03-29T04:10:23.311930+0000","last_clean":"2024-03-29T04:10:23.311930+0000","last_became_active":"2024-03-29T04:10:23.308430+0000","last_became_peered":"2024-03-29T04:10:23.308430+0000","last_unstale":"2024-03-29T04:10:23.311930+0000","last_undegraded":"2024-03-29T04:10:23.311930+0000","last_fullsized":"2024-03-29T04:10:23.311930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307389+0000","last_change":"2024-03-29T04:10:22.306797+0000","last_active":"2024-03-29T04:10:22.307389+0000","last_peered":"2024-03-29T04:10:22.307389+0000","last_clean":"2024-03-29T04:10:22.307389+0000","last_became_active":"2024-03-29T04:10:22.306504+0000","last_became_peered":"2024-03-29T04:10:22.306504+0000","last_unstale":"2024-03-29T04:10:22.307389+0000","last_undegraded":"2024-03-29T04:10:22.307389+0000","last_fullsized":"2024-03-29T04:10:22.307389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320184+0000","last_change":"2024-03-29T04:10:23.314380+0000","last_active":"2024-03-29T04:10:23.320184+0000","last_peered":"2024-03-29T04:10:23.320184+0000","last_clean":"2024-03-29T04:10:23.320184+0000","last_became_active":"2024-03-29T04:10:23.314129+0000","last_became_peered":"2024-03-29T04:10:23.314129+0000","last_unstale":"2024-03-29T04:10:23.320184+0000","last_undegraded":"2024-03-29T04:10:23.320184+0000","last_fullsized":"2024-03-29T04:10:23.320184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.311633+0000","last_change":"2024-03-29T04:10:22.310915+0000","last_active":"2024-03-29T04:10:22.311633+0000","last_peered":"2024-03-29T04:10:22.311633+0000","last_clean":"2024-03-29T04:10:22.311633+0000","last_became_active":"2024-03-29T04:10:22.310630+0000","last_became_peered":"2024-03-29T04:10:22.310630+0000","last_unstale":"2024-03-29T04:10:22.311633+0000","last_undegraded":"2024-03-29T04:10:22.311633+0000","last_fullsized":"2024-03-29T04:10:22.311633+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301511+0000","last_change":"2024-03-29T04:10:22.301511+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301511+0000","last_undegraded":"2024-03-29T04:10:22.301511+0000","last_fullsized":"2024-03-29T04:10:22.301511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.319869+0000","last_change":"2024-03-29T04:10:22.304262+0000","last_active":"2024-03-29T04:10:23.319869+0000","last_peered":"2024-03-29T04:10:23.319869+0000","last_clean":"2024-03-29T04:10:23.319869+0000","last_became_active":"2024-03-29T04:10:22.303999+0000","last_became_peered":"2024-03-29T04:10:22.303999+0000","last_unstale":"2024-03-29T04:10:23.319869+0000","last_undegraded":"2024-03-29T04:10:23.319869+0000","last_fullsized":"2024-03-29T04:10:23.319869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292420+0000","last_change":"2024-03-29T04:10:22.292420+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292420+0000","last_undegraded":"2024-03-29T04:10:22.292420+0000","last_fullsized":"2024-03-29T04:10:22.292420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295533+0000","last_change":"2024-03-29T04:10:22.307417+0000","last_active":"2024-03-29T04:10:23.295533+0000","last_peered":"2024-03-29T04:10:23.295533+0000","last_clean":"2024-03-29T04:10:23.295533+0000","last_became_active":"2024-03-29T04:10:22.307123+0000","last_became_peered":"2024-03-29T04:10:22.307123+0000","last_unstale":"2024-03-29T04:10:23.295533+0000","last_undegraded":"2024-03-29T04:10:23.295533+0000","last_fullsized":"2024-03-29T04:10:23.295533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312368+0000","last_change":"2024-03-29T04:10:23.308517+0000","last_active":"2024-03-29T04:10:23.312368+0000","last_peered":"2024-03-29T04:10:23.312368+0000","last_clean":"2024-03-29T04:10:23.312368+0000","last_became_active":"2024-03-29T04:10:23.308180+0000","last_became_peered":"2024-03-29T04:10:23.308180+0000","last_unstale":"2024-03-29T04:10:23.312368+0000","last_undegraded":"2024-03-29T04:10:23.312368+0000","last_fullsized":"2024-03-29T04:10:23.312368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.296007+0000","last_change":"2024-03-29T04:10:22.294851+0000","last_active":"2024-03-29T04:10:22.296007+0000","last_peered":"2024-03-29T04:10:22.296007+0000","last_clean":"2024-03-29T04:10:22.296007+0000","last_became_active":"2024-03-29T04:10:22.294580+0000","last_became_peered":"2024-03-29T04:10:22.294580+0000","last_unstale":"2024-03-29T04:10:22.296007+0000","last_undegraded":"2024-03-29T04:10:22.296007+0000","last_fullsized":"2024-03-29T04:10:22.296007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.321112+0000","last_change":"2024-03-29T04:10:23.313712+0000","last_active":"2024-03-29T04:10:23.321112+0000","last_peered":"2024-03-29T04:10:23.321112+0000","last_clean":"2024-03-29T04:10:23.321112+0000","last_became_active":"2024-03-29T04:10:23.313467+0000","last_became_peered":"2024-03-29T04:10:23.313467+0000","last_unstale":"2024-03-29T04:10:23.321112+0000","last_undegraded":"2024-03-29T04:10:23.321112+0000","last_fullsized":"2024-03-29T04:10:23.321112+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:39:13.669270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295738+0000","last_change":"2024-03-29T04:10:22.305312+0000","last_active":"2024-03-29T04:10:23.295738+0000","last_peered":"2024-03-29T04:10:23.295738+0000","last_clean":"2024-03-29T04:10:23.295738+0000","last_became_active":"2024-03-29T04:10:22.304984+0000","last_became_peered":"2024-03-29T04:10:22.304984+0000","last_unstale":"2024-03-29T04:10:23.295738+0000","last_undegraded":"2024-03-29T04:10:23.295738+0000","last_fullsized":"2024-03-29T04:10:23.295738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:35.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314641+0000","last_change":"2024-03-29T04:10:23.304934+0000","last_active":"2024-03-29T04:10:23.314641+0000","last_peered":"2024-03-29T04:10:23.314641+0000","last_clean":"2024-03-29T04:10:23.314641+0000","last_became_active":"2024-03-29T04:10:23.304669+0000","last_became_peered":"2024-03-29T04:10:23.304669+0000","last_unstale":"2024-03-29T04:10:23.314641+0000","last_undegraded":"2024-03-29T04:10:23.314641+0000","last_fullsized":"2024-03-29T04:10:23.314641+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309574+0000","last_change":"2024-03-29T04:10:22.308050+0000","last_active":"2024-03-29T04:10:22.309574+0000","last_peered":"2024-03-29T04:10:22.309574+0000","last_clean":"2024-03-29T04:10:22.309574+0000","last_became_active":"2024-03-29T04:10:22.307753+0000","last_became_peered":"2024-03-29T04:10:22.307753+0000","last_unstale":"2024-03-29T04:10:22.309574+0000","last_undegraded":"2024-03-29T04:10:22.309574+0000","last_fullsized":"2024-03-29T04:10:22.309574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322352+0000","last_change":"2024-03-29T04:10:23.317220+0000","last_active":"2024-03-29T04:10:23.322352+0000","last_peered":"2024-03-29T04:10:23.322352+0000","last_clean":"2024-03-29T04:10:23.322352+0000","last_became_active":"2024-03-29T04:10:23.316962+0000","last_became_peered":"2024-03-29T04:10:23.316962+0000","last_unstale":"2024-03-29T04:10:23.322352+0000","last_undegraded":"2024-03-29T04:10:23.322352+0000","last_fullsized":"2024-03-29T04:10:23.322352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:44.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310684+0000","last_change":"2024-03-29T04:10:22.308995+0000","last_active":"2024-03-29T04:10:22.310684+0000","last_peered":"2024-03-29T04:10:22.310684+0000","last_clean":"2024-03-29T04:10:22.310684+0000","last_became_active":"2024-03-29T04:10:22.308679+0000","last_became_peered":"2024-03-29T04:10:22.308679+0000","last_unstale":"2024-03-29T04:10:22.310684+0000","last_undegraded":"2024-03-29T04:10:22.310684+0000","last_fullsized":"2024-03-29T04:10:22.310684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292102+0000","last_change":"2024-03-29T04:10:22.292102+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292102+0000","last_undegraded":"2024-03-29T04:10:22.292102+0000","last_fullsized":"2024-03-29T04:10:22.292102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.323985+0000","last_change":"2024-03-29T04:10:22.306679+0000","last_active":"2024-03-29T04:10:23.323985+0000","last_peered":"2024-03-29T04:10:23.323985+0000","last_clean":"2024-03-29T04:10:23.323985+0000","last_became_active":"2024-03-29T04:10:22.306411+0000","last_became_peered":"2024-03-29T04:10:22.306411+0000","last_unstale":"2024-03-29T04:10:23.323985+0000","last_undegraded":"2024-03-29T04:10:23.323985+0000","last_fullsized":"2024-03-29T04:10:23.323985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:22.259732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322368+0000","last_change":"2024-03-29T04:10:23.316218+0000","last_active":"2024-03-29T04:10:23.322368+0000","last_peered":"2024-03-29T04:10:23.322368+0000","last_clean":"2024-03-29T04:10:23.322368+0000","last_became_active":"2024-03-29T04:10:23.315899+0000","last_became_peered":"2024-03-29T04:10:23.315899+0000","last_unstale":"2024-03-29T04:10:23.322368+0000","last_undegraded":"2024-03-29T04:10:23.322368+0000","last_fullsized":"2024-03-29T04:10:23.322368+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322565+0000","last_change":"2024-03-29T04:10:22.310885+0000","last_active":"2024-03-29T04:10:23.322565+0000","last_peered":"2024-03-29T04:10:23.322565+0000","last_clean":"2024-03-29T04:10:23.322565+0000","last_became_active":"2024-03-29T04:10:22.310589+0000","last_became_peered":"2024-03-29T04:10:22.310589+0000","last_unstale":"2024-03-29T04:10:23.322565+0000","last_undegraded":"2024-03-29T04:10:23.322565+0000","last_fullsized":"2024-03-29T04:10:23.322565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:00.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317792+0000","last_change":"2024-03-29T04:10:23.312331+0000","last_active":"2024-03-29T04:10:23.317792+0000","last_peered":"2024-03-29T04:10:23.317792+0000","last_clean":"2024-03-29T04:10:23.317792+0000","last_became_active":"2024-03-29T04:10:23.311961+0000","last_became_peered":"2024-03-29T04:10:23.311961+0000","last_unstale":"2024-03-29T04:10:23.317792+0000","last_undegraded":"2024-03-29T04:10:23.317792+0000","last_fullsized":"2024-03-29T04:10:23.317792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.298233+0000","last_change":"2024-03-29T04:10:22.308119+0000","last_active":"2024-03-29T04:10:23.298233+0000","last_peered":"2024-03-29T04:10:23.298233+0000","last_clean":"2024-03-29T04:10:23.298233+0000","last_became_active":"2024-03-29T04:10:22.307849+0000","last_became_peered":"2024-03-29T04:10:22.307849+0000","last_unstale":"2024-03-29T04:10:23.298233+0000","last_undegraded":"2024-03-29T04:10:23.298233+0000","last_fullsized":"2024-03-29T04:10:23.298233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316478+0000","last_change":"2024-03-29T04:10:23.310446+0000","last_active":"2024-03-29T04:10:23.316478+0000","last_peered":"2024-03-29T04:10:23.316478+0000","last_clean":"2024-03-29T04:10:23.316478+0000","last_became_active":"2024-03-29T04:10:23.310183+0000","last_became_peered":"2024-03-29T04:10:23.310183+0000","last_unstale":"2024-03-29T04:10:23.316478+0000","last_undegraded":"2024-03-29T04:10:23.316478+0000","last_fullsized":"2024-03-29T04:10:23.316478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.291912+0000","last_change":"2024-03-29T04:10:22.297317+0000","last_active":"2024-03-29T04:10:23.291912+0000","last_peered":"2024-03-29T04:10:23.291912+0000","last_clean":"2024-03-29T04:10:23.291912+0000","last_became_active":"2024-03-29T04:10:22.297044+0000","last_became_peered":"2024-03-29T04:10:22.297044+0000","last_unstale":"2024-03-29T04:10:23.291912+0000","last_undegraded":"2024-03-29T04:10:23.291912+0000","last_fullsized":"2024-03-29T04:10:23.291912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:56.164692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317543+0000","last_change":"2024-03-29T04:10:23.312644+0000","last_active":"2024-03-29T04:10:23.317543+0000","last_peered":"2024-03-29T04:10:23.317543+0000","last_clean":"2024-03-29T04:10:23.317543+0000","last_became_active":"2024-03-29T04:10:23.312370+0000","last_became_peered":"2024-03-29T04:10:23.312370+0000","last_unstale":"2024-03-29T04:10:23.317543+0000","last_undegraded":"2024-03-29T04:10:23.317543+0000","last_fullsized":"2024-03-29T04:10:23.317543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320581+0000","last_change":"2024-03-29T04:10:22.304877+0000","last_active":"2024-03-29T04:10:23.320581+0000","last_peered":"2024-03-29T04:10:23.320581+0000","last_clean":"2024-03-29T04:10:23.320581+0000","last_became_active":"2024-03-29T04:10:22.304573+0000","last_became_peered":"2024-03-29T04:10:22.304573+0000","last_unstale":"2024-03-29T04:10:23.320581+0000","last_undegraded":"2024-03-29T04:10:23.320581+0000","last_fullsized":"2024-03-29T04:10:23.320581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.315960+0000","last_change":"2024-03-29T04:10:23.309896+0000","last_active":"2024-03-29T04:10:23.315960+0000","last_peered":"2024-03-29T04:10:23.315960+0000","last_clean":"2024-03-29T04:10:23.315960+0000","last_became_active":"2024-03-29T04:10:23.308057+0000","last_became_peered":"2024-03-29T04:10:23.308057+0000","last_unstale":"2024-03-29T04:10:23.315960+0000","last_undegraded":"2024-03-29T04:10:23.315960+0000","last_fullsized":"2024-03-29T04:10:23.315960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.307929+0000","last_change":"2024-03-29T04:10:22.307155+0000","last_active":"2024-03-29T04:10:22.307929+0000","last_peered":"2024-03-29T04:10:22.307929+0000","last_clean":"2024-03-29T04:10:22.307929+0000","last_became_active":"2024-03-29T04:10:22.306817+0000","last_became_peered":"2024-03-29T04:10:22.306817+0000","last_unstale":"2024-03-29T04:10:22.307929+0000","last_undegraded":"2024-03-29T04:10:22.307929+0000","last_fullsized":"2024-03-29T04:10:22.307929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:10.707379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301030+0000","last_change":"2024-03-29T04:10:22.301030+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301030+0000","last_undegraded":"2024-03-29T04:10:22.301030+0000","last_fullsized":"2024-03-29T04:10:22.301030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.309562+0000","last_change":"2024-03-29T04:10:22.307690+0000","last_active":"2024-03-29T04:10:22.309562+0000","last_peered":"2024-03-29T04:10:22.309562+0000","last_clean":"2024-03-29T04:10:22.309562+0000","last_became_active":"2024-03-29T04:10:22.307424+0000","last_became_peered":"2024-03-29T04:10:22.307424+0000","last_unstale":"2024-03-29T04:10:22.309562+0000","last_undegraded":"2024-03-29T04:10:22.309562+0000","last_fullsized":"2024-03-29T04:10:22.309562+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292383+0000","last_change":"2024-03-29T04:10:22.292383+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292383+0000","last_undegraded":"2024-03-29T04:10:22.292383+0000","last_fullsized":"2024-03-29T04:10:22.292383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.318361+0000","last_change":"2024-03-29T04:10:22.303744+0000","last_active":"2024-03-29T04:10:23.318361+0000","last_peered":"2024-03-29T04:10:23.318361+0000","last_clean":"2024-03-29T04:10:23.318361+0000","last_became_active":"2024-03-29T04:10:22.303477+0000","last_became_peered":"2024-03-29T04:10:22.303477+0000","last_unstale":"2024-03-29T04:10:23.318361+0000","last_undegraded":"2024-03-29T04:10:23.318361+0000","last_fullsized":"2024-03-29T04:10:23.318361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:03:07.972943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.310284+0000","last_change":"2024-03-29T04:10:23.306990+0000","last_active":"2024-03-29T04:10:23.310284+0000","last_peered":"2024-03-29T04:10:23.310284+0000","last_clean":"2024-03-29T04:10:23.310284+0000","last_became_active":"2024-03-29T04:10:23.306635+0000","last_became_peered":"2024-03-29T04:10:23.306635+0000","last_unstale":"2024-03-29T04:10:23.310284+0000","last_undegraded":"2024-03-29T04:10:23.310284+0000","last_fullsized":"2024-03-29T04:10:23.310284+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:32:25.613005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297872+0000","last_change":"2024-03-29T04:10:22.300528+0000","last_active":"2024-03-29T04:10:23.297872+0000","last_peered":"2024-03-29T04:10:23.297872+0000","last_clean":"2024-03-29T04:10:23.297872+0000","last_became_active":"2024-03-29T04:10:22.299874+0000","last_became_peered":"2024-03-29T04:10:22.299874+0000","last_unstale":"2024-03-29T04:10:23.297872+0000","last_undegraded":"2024-03-29T04:10:23.297872+0000","last_fullsized":"2024-03-29T04:10:23.297872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316810+0000","last_change":"2024-03-29T04:10:23.311796+0000","last_active":"2024-03-29T04:10:23.316810+0000","last_peered":"2024-03-29T04:10:23.316810+0000","last_clean":"2024-03-29T04:10:23.316810+0000","last_became_active":"2024-03-29T04:10:23.311351+0000","last_became_peered":"2024-03-29T04:10:23.311351+0000","last_unstale":"2024-03-29T04:10:23.316810+0000","last_undegraded":"2024-03-29T04:10:23.316810+0000","last_fullsized":"2024-03-29T04:10:23.316810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:11.170179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.309583+0000","last_change":"2024-03-29T04:10:22.308134+0000","last_active":"2024-03-29T04:10:23.309583+0000","last_peered":"2024-03-29T04:10:23.309583+0000","last_clean":"2024-03-29T04:10:23.309583+0000","last_became_active":"2024-03-29T04:10:22.307860+0000","last_became_peered":"2024-03-29T04:10:22.307860+0000","last_unstale":"2024-03-29T04:10:23.309583+0000","last_undegraded":"2024-03-29T04:10:23.309583+0000","last_fullsized":"2024-03-29T04:10:23.309583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:20:17.587318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.311340+0000","last_change":"2024-03-29T04:10:23.309927+0000","last_active":"2024-03-29T04:10:23.311340+0000","last_peered":"2024-03-29T04:10:23.311340+0000","last_clean":"2024-03-29T04:10:23.311340+0000","last_became_active":"2024-03-29T04:10:23.309623+0000","last_became_peered":"2024-03-29T04:10:23.309623+0000","last_unstale":"2024-03-29T04:10:23.311340+0000","last_undegraded":"2024-03-29T04:10:23.311340+0000","last_fullsized":"2024-03-29T04:10:23.311340+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.301262+0000","last_change":"2024-03-29T04:10:22.297913+0000","last_active":"2024-03-29T04:10:23.301262+0000","last_peered":"2024-03-29T04:10:23.301262+0000","last_clean":"2024-03-29T04:10:23.301262+0000","last_became_active":"2024-03-29T04:10:22.296015+0000","last_became_peered":"2024-03-29T04:10:22.296015+0000","last_unstale":"2024-03-29T04:10:23.301262+0000","last_undegraded":"2024-03-29T04:10:23.301262+0000","last_fullsized":"2024-03-29T04:10:23.301262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:13.154992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.300580+0000","last_change":"2024-03-29T04:10:22.300580+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.300580+0000","last_undegraded":"2024-03-29T04:10:22.300580+0000","last_fullsized":"2024-03-29T04:10:22.300580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292907+0000","last_change":"2024-03-29T04:10:22.305287+0000","last_active":"2024-03-29T04:10:23.292907+0000","last_peered":"2024-03-29T04:10:23.292907+0000","last_clean":"2024-03-29T04:10:23.292907+0000","last_became_active":"2024-03-29T04:10:22.304999+0000","last_became_peered":"2024-03-29T04:10:22.304999+0000","last_unstale":"2024-03-29T04:10:23.292907+0000","last_undegraded":"2024-03-29T04:10:23.292907+0000","last_fullsized":"2024-03-29T04:10:23.292907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.312347+0000","last_change":"2024-03-29T04:10:23.307638+0000","last_active":"2024-03-29T04:10:23.312347+0000","last_peered":"2024-03-29T04:10:23.312347+0000","last_clean":"2024-03-29T04:10:23.312347+0000","last_became_active":"2024-03-29T04:10:23.307381+0000","last_became_peered":"2024-03-29T04:10:23.307381+0000","last_unstale":"2024-03-29T04:10:23.312347+0000","last_undegraded":"2024-03-29T04:10:23.312347+0000","last_fullsized":"2024-03-29T04:10:23.312347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299841+0000","last_change":"2024-03-29T04:10:22.307254+0000","last_active":"2024-03-29T04:10:23.299841+0000","last_peered":"2024-03-29T04:10:23.299841+0000","last_clean":"2024-03-29T04:10:23.299841+0000","last_became_active":"2024-03-29T04:10:22.306976+0000","last_became_peered":"2024-03-29T04:10:22.306976+0000","last_unstale":"2024-03-29T04:10:23.299841+0000","last_undegraded":"2024-03-29T04:10:23.299841+0000","last_fullsized":"2024-03-29T04:10:23.299841+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:54:11.970574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.320128+0000","last_change":"2024-03-29T04:10:23.317609+0000","last_active":"2024-03-29T04:10:23.320128+0000","last_peered":"2024-03-29T04:10:23.320128+0000","last_clean":"2024-03-29T04:10:23.320128+0000","last_became_active":"2024-03-29T04:10:23.317364+0000","last_became_peered":"2024-03-29T04:10:23.317364+0000","last_unstale":"2024-03-29T04:10:23.320128+0000","last_undegraded":"2024-03-29T04:10:23.320128+0000","last_fullsized":"2024-03-29T04:10:23.320128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.310663+0000","last_change":"2024-03-29T04:10:22.308920+0000","last_active":"2024-03-29T04:10:22.310663+0000","last_peered":"2024-03-29T04:10:22.310663+0000","last_clean":"2024-03-29T04:10:22.310663+0000","last_became_active":"2024-03-29T04:10:22.308648+0000","last_became_peered":"2024-03-29T04:10:22.308648+0000","last_unstale":"2024-03-29T04:10:22.310663+0000","last_undegraded":"2024-03-29T04:10:22.310663+0000","last_fullsized":"2024-03-29T04:10:22.310663+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314664+0000","last_change":"2024-03-29T04:10:23.305533+0000","last_active":"2024-03-29T04:10:23.314664+0000","last_peered":"2024-03-29T04:10:23.314664+0000","last_clean":"2024-03-29T04:10:23.314664+0000","last_became_active":"2024-03-29T04:10:23.305295+0000","last_became_peered":"2024-03-29T04:10:23.305295+0000","last_unstale":"2024-03-29T04:10:23.314664+0000","last_undegraded":"2024-03-29T04:10:23.314664+0000","last_fullsized":"2024-03-29T04:10:23.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.304196+0000","last_change":"2024-03-29T04:10:22.309267+0000","last_active":"2024-03-29T04:10:23.304196+0000","last_peered":"2024-03-29T04:10:23.304196+0000","last_clean":"2024-03-29T04:10:23.304196+0000","last_became_active":"2024-03-29T04:10:22.308996+0000","last_became_peered":"2024-03-29T04:10:22.308996+0000","last_unstale":"2024-03-29T04:10:23.304196+0000","last_undegraded":"2024-03-29T04:10:23.304196+0000","last_fullsized":"2024-03-29T04:10:23.304196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:27.354497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.317732+0000","last_change":"2024-03-29T04:10:23.313551+0000","last_active":"2024-03-29T04:10:23.317732+0000","last_peered":"2024-03-29T04:10:23.317732+0000","last_clean":"2024-03-29T04:10:23.317732+0000","last_became_active":"2024-03-29T04:10:23.313289+0000","last_became_peered":"2024-03-29T04:10:23.313289+0000","last_unstale":"2024-03-29T04:10:23.317732+0000","last_undegraded":"2024-03-29T04:10:23.317732+0000","last_fullsized":"2024-03-29T04:10:23.317732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.299143+0000","last_change":"2024-03-29T04:10:22.308648+0000","last_active":"2024-03-29T04:10:23.299143+0000","last_peered":"2024-03-29T04:10:23.299143+0000","last_clean":"2024-03-29T04:10:23.299143+0000","last_became_active":"2024-03-29T04:10:22.308334+0000","last_became_peered":"2024-03-29T04:10:22.308334+0000","last_unstale":"2024-03-29T04:10:23.299143+0000","last_undegraded":"2024-03-29T04:10:23.299143+0000","last_fullsized":"2024-03-29T04:10:23.299143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.322380+0000","last_change":"2024-03-29T04:10:23.319502+0000","last_active":"2024-03-29T04:10:23.322380+0000","last_peered":"2024-03-29T04:10:23.322380+0000","last_clean":"2024-03-29T04:10:23.322380+0000","last_became_active":"2024-03-29T04:10:23.319209+0000","last_became_peered":"2024-03-29T04:10:23.319209+0000","last_unstale":"2024-03-29T04:10:23.322380+0000","last_undegraded":"2024-03-29T04:10:23.322380+0000","last_fullsized":"2024-03-29T04:10:23.322380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:36.568021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.306085+0000","last_change":"2024-03-29T04:10:22.307995+0000","last_active":"2024-03-29T04:10:23.306085+0000","last_peered":"2024-03-29T04:10:23.306085+0000","last_clean":"2024-03-29T04:10:23.306085+0000","last_became_active":"2024-03-29T04:10:22.307723+0000","last_became_peered":"2024-03-29T04:10:22.307723+0000","last_unstale":"2024-03-29T04:10:23.306085+0000","last_undegraded":"2024-03-29T04:10:23.306085+0000","last_fullsized":"2024-03-29T04:10:23.306085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.316387+0000","last_change":"2024-03-29T04:10:23.308586+0000","last_active":"2024-03-29T04:10:23.316387+0000","last_peered":"2024-03-29T04:10:23.316387+0000","last_clean":"2024-03-29T04:10:23.316387+0000","last_became_active":"2024-03-29T04:10:23.308278+0000","last_became_peered":"2024-03-29T04:10:23.308278+0000","last_unstale":"2024-03-29T04:10:23.316387+0000","last_undegraded":"2024-03-29T04:10:23.316387+0000","last_fullsized":"2024-03-29T04:10:23.316387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:08.978931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.295435+0000","last_change":"2024-03-29T04:10:22.309876+0000","last_active":"2024-03-29T04:10:23.295435+0000","last_peered":"2024-03-29T04:10:23.295435+0000","last_clean":"2024-03-29T04:10:23.295435+0000","last_became_active":"2024-03-29T04:10:22.309473+0000","last_became_peered":"2024-03-29T04:10:22.309473+0000","last_unstale":"2024-03-29T04:10:23.295435+0000","last_undegraded":"2024-03-29T04:10:23.295435+0000","last_fullsized":"2024-03-29T04:10:23.295435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:20.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.292868+0000","last_change":"2024-03-29T04:10:22.292868+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.292868+0000","last_undegraded":"2024-03-29T04:10:22.292868+0000","last_fullsized":"2024-03-29T04:10:22.292868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-29T04:10:22.303802+0000","last_change":"2024-03-29T04:10:22.302187+0000","last_active":"2024-03-29T04:10:22.303802+0000","last_peered":"2024-03-29T04:10:22.303802+0000","last_clean":"2024-03-29T04:10:22.303802+0000","last_became_active":"2024-03-29T04:10:22.301830+0000","last_became_peered":"2024-03-29T04:10:22.301830+0000","last_unstale":"2024-03-29T04:10:22.303802+0000","last_undegraded":"2024-03-29T04:10:22.303802+0000","last_fullsized":"2024-03-29T04:10:22.303802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:07.597145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.314171+0000","last_change":"2024-03-29T04:10:23.303127+0000","last_active":"2024-03-29T04:10:23.314171+0000","last_peered":"2024-03-29T04:10:23.314171+0000","last_clean":"2024-03-29T04:10:23.314171+0000","last_became_active":"2024-03-29T04:10:23.302882+0000","last_became_peered":"2024-03-29T04:10:23.302882+0000","last_unstale":"2024-03-29T04:10:23.314171+0000","last_undegraded":"2024-03-29T04:10:23.314171+0000","last_fullsized":"2024-03-29T04:10:23.314171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.297157+0000","last_change":"2024-03-29T04:10:22.304632+0000","last_active":"2024-03-29T04:10:23.297157+0000","last_peered":"2024-03-29T04:10:23.297157+0000","last_clean":"2024-03-29T04:10:23.297157+0000","last_became_active":"2024-03-29T04:10:22.304363+0000","last_became_peered":"2024-03-29T04:10:22.304363+0000","last_unstale":"2024-03-29T04:10:23.297157+0000","last_undegraded":"2024-03-29T04:10:23.297157+0000","last_fullsized":"2024-03-29T04:10:23.297157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:35.091291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-03-29T04:10:22.301182+0000","last_change":"2024-03-29T04:10:22.301182+0000","last_active":"2024-03-29T04:10:22.281232+0000","last_peered":"2024-03-29T04:10:22.281232+0000","last_clean":"2024-03-29T04:10:22.281232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T04:10:22.301182+0000","last_undegraded":"2024-03-29T04:10:22.301182+0000","last_fullsized":"2024-03-29T04:10:22.301182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.293779+0000","last_change":"2024-03-29T04:10:22.310649+0000","last_active":"2024-03-29T04:10:23.293779+0000","last_peered":"2024-03-29T04:10:23.293779+0000","last_clean":"2024-03-29T04:10:23.293779+0000","last_became_active":"2024-03-29T04:10:22.310382+0000","last_became_peered":"2024-03-29T04:10:22.310382+0000","last_unstale":"2024-03-29T04:10:23.293779+0000","last_undegraded":"2024-03-29T04:10:23.293779+0000","last_fullsized":"2024-03-29T04:10:23.293779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T04:10:23.292459+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:23.292459+0000","last_peered":"2024-03-29T04:10:23.292459+0000","last_clean":"2024-03-29T04:10:23.292459+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:23.292459+0000","last_undegraded":"2024-03-29T04:10:23.292459+0000","last_fullsized":"2024-03-29T04:10:23.292459+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":192127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:28.027 INFO:tasks.ceph.mds.a.smithi181.stderr:2024-03-29T04:10:28.048+0000 7f74abe36640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T04:10:30.936 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:10:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.095+0000 7facf8d59640 1 Processor -- start 2024-03-29T04:10:31.075 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.095+0000 7facf8d59640 1 -- start start 2024-03-29T04:10:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf8d59640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf415a960 0x7facf4071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf8d59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf40cd3b0 0x7facf40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf8d59640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf40724e0 con 0x7facf4071fb0 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf8d59640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7facf40730f0 con 0x7facf415a960 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf8d59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7facf40727c0 con 0x7facf40cd3b0 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf415a960 0x7facf4071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facf415a960 0x7facf4071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39582/0 (socket says 172.21.15.181:39582) 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 -- 172.21.15.181:0/3078917173 learned_addr learned my addr 172.21.15.181:0/3078917173 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:31.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf1d9b640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf40cd3b0 0x7facf40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 -- 172.21.15.181:0/3078917173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf40cd3b0 msgr2=0x7facf40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf40cd3b0 0x7facf40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 -- 172.21.15.181:0/3078917173 >> v1:172.21.15.187:6789/0 conn(0x7facf4071fb0 legacy=0x7facf415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.096+0000 7facf259c640 1 -- 172.21.15.181:0/3078917173 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf4072aa0 con 0x7facf415a960 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.097+0000 7facf259c640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf4071a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7facec009e50 tx=0x7facec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6522951fee85408 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.097+0000 7facf159a640 1 -- 172.21.15.181:0/3078917173 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec02ca70 con 0x7facf415a960 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.097+0000 7facf159a640 1 -- 172.21.15.181:0/3078917173 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facec02cc10 con 0x7facf415a960 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.097+0000 7facf159a640 1 -- 172.21.15.181:0/3078917173 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec0343f0 con 0x7facf415a960 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 -- 172.21.15.181:0/3078917173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 msgr2=0x7facf4071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf4071a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7facec009e50 tx=0x7facec02bed0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 -- 172.21.15.181:0/3078917173 shutdown_connections 2024-03-29T04:10:31.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf40cd3b0 0x7facf40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 --2- 172.21.15.181:0/3078917173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf4071a70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 -- 172.21.15.181:0/3078917173 >> 172.21.15.181:0/3078917173 conn(0x7facf40b1c90 msgr2=0x7facf40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.089 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 -- 172.21.15.181:0/3078917173 shutdown_connections 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.098+0000 7facf8d59640 1 -- 172.21.15.181:0/3078917173 wait complete. 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.100+0000 7facf8d59640 1 Processor -- start 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.100+0000 7facf8d59640 1 -- start start 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf4071fb0 0x7facf414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 0x7facf414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7facf40cede0 con 0x7facf415a960 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7facf415ad40 con 0x7facf40cd3b0 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.101+0000 7facf8d59640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7facf40cef60 con 0x7facf4071fb0 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39586/0 (socket says 172.21.15.181:39586) 2024-03-29T04:10:31.091 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 -- 172.21.15.181:0/488012665 learned_addr learned my addr 172.21.15.181:0/488012665 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf1d9b640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 0x7facf414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf259c640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf4071fb0 0x7facf414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 -- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf4071fb0 msgr2=0x7facf414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf4071fb0 0x7facf414e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 -- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 msgr2=0x7facf414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 0x7facf414ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 -- 172.21.15.181:0/488012665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf4151ff0 con 0x7facf415a960 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf1d9b640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 0x7facf414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf2d9d640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7face400c3e0 tx=0x7face400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011790 con 0x7facf415a960 2024-03-29T04:10:31.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4017070 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf4153b60 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011bb0 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.102+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf4151510 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.103+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf4075a10 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.104+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7face401c460 con 0x7facf415a960 2024-03-29T04:10:31.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.104+0000 7faceaffd640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 0x7facd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.104+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7face408b550 con 0x7facf415a960 2024-03-29T04:10:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.104+0000 7facf259c640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 0x7facd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.105+0000 7facf259c640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 0x7facd406ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facec035000 tx=0x7facec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.113+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7face4059a70 con 0x7facf415a960 2024-03-29T04:10:31.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.290+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facf41545f0 con 0x7facd406c560 2024-03-29T04:10:31.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.296+0000 7faceaffd640 1 -- 172.21.15.181:0/488012665 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251899 (secure 0 0 0) 0x7facf41545f0 con 0x7facd406c560 2024-03-29T04:10:31.278 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:31.280 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:10:31.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 msgr2=0x7facd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 0x7facd406ea20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facec035000 tx=0x7facec030000 comp rx=0 tx=0).stop 2024-03-29T04:10:31.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 msgr2=0x7facf41f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7face400c3e0 tx=0x7face400c8a0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 shutdown_connections 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facf415a960 0x7facf41f47c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facf40cd3b0 0x7facf414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facd406c560 0x7facd406ea20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 --2- 172.21.15.181:0/488012665 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7facf4071fb0 0x7facf414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 >> 172.21.15.181:0/488012665 conn(0x7facf40b1c90 msgr2=0x7facf40cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 shutdown_connections 2024-03-29T04:10:31.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.300+0000 7facf8d59640 1 -- 172.21.15.181:0/488012665 wait complete. 2024-03-29T04:10:31.292 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-29T04:10:29.587595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220940,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753780,"statfs":{"total":773094113280,"available":772867870720,"internally_reserved":0,"allocated":4857856,"data_stored":2762940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.299215"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.304225+0000","last_change":"2024-03-29T04:10:23.313049+0000","last_active":"2024-03-29T04:10:24.304225+0000","last_peered":"2024-03-29T04:10:24.304225+0000","last_clean":"2024-03-29T04:10:24.304225+0000","last_became_active":"2024-03-29T04:10:23.312619+0000","last_became_peered":"2024-03-29T04:10:23.312619+0000","last_unstale":"2024-03-29T04:10:24.304225+0000","last_undegraded":"2024-03-29T04:10:24.304225+0000","last_fullsized":"2024-03-29T04:10:24.304225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:21:55.332701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.304114+0000","last_change":"2024-03-29T04:10:23.304958+0000","last_active":"2024-03-29T04:10:24.304114+0000","last_peered":"2024-03-29T04:10:24.304114+0000","last_clean":"2024-03-29T04:10:24.304114+0000","last_became_active":"2024-03-29T04:10:23.304714+0000","last_became_peered":"2024-03-29T04:10:23.304714+0000","last_unstale":"2024-03-29T04:10:24.304114+0000","last_undegraded":"2024-03-29T04:10:24.304114+0000","last_fullsized":"2024-03-29T04:10:24.304114+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:42.703771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362434+0000","last_change":"2024-03-29T04:10:23.315527+0000","last_active":"2024-03-29T04:10:25.362434+0000","last_peered":"2024-03-29T04:10:25.362434+0000","last_clean":"2024-03-29T04:10:25.362434+0000","last_became_active":"2024-03-29T04:10:23.315242+0000","last_became_peered":"2024-03-29T04:10:23.315242+0000","last_unstale":"2024-03-29T04:10:25.362434+0000","last_undegraded":"2024-03-29T04:10:25.362434+0000","last_fullsized":"2024-03-29T04:10:25.362434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295901+0000","last_change":"2024-03-29T04:10:23.315865+0000","last_active":"2024-03-29T04:10:25.295901+0000","last_peered":"2024-03-29T04:10:25.295901+0000","last_clean":"2024-03-29T04:10:25.295901+0000","last_became_active":"2024-03-29T04:10:23.315580+0000","last_became_peered":"2024-03-29T04:10:23.315580+0000","last_unstale":"2024-03-29T04:10:25.295901+0000","last_undegraded":"2024-03-29T04:10:25.295901+0000","last_fullsized":"2024-03-29T04:10:25.295901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:51:35.316362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295697+0000","last_change":"2024-03-29T04:10:23.315150+0000","last_active":"2024-03-29T04:10:25.295697+0000","last_peered":"2024-03-29T04:10:25.295697+0000","last_clean":"2024-03-29T04:10:25.295697+0000","last_became_active":"2024-03-29T04:10:23.314879+0000","last_became_peered":"2024-03-29T04:10:23.314879+0000","last_unstale":"2024-03-29T04:10:25.295697+0000","last_undegraded":"2024-03-29T04:10:25.295697+0000","last_fullsized":"2024-03-29T04:10:25.295697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:57.166999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487740+0000","last_change":"2024-03-29T04:10:23.311220+0000","last_active":"2024-03-29T04:10:24.487740+0000","last_peered":"2024-03-29T04:10:24.487740+0000","last_clean":"2024-03-29T04:10:24.487740+0000","last_became_active":"2024-03-29T04:10:23.310727+0000","last_became_peered":"2024-03-29T04:10:23.310727+0000","last_unstale":"2024-03-29T04:10:24.487740+0000","last_undegraded":"2024-03-29T04:10:24.487740+0000","last_fullsized":"2024-03-29T04:10:24.487740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.297891+0000","last_change":"2024-03-29T04:10:23.313509+0000","last_active":"2024-03-29T04:10:25.297891+0000","last_peered":"2024-03-29T04:10:25.297891+0000","last_clean":"2024-03-29T04:10:25.297891+0000","last_became_active":"2024-03-29T04:10:23.313225+0000","last_became_peered":"2024-03-29T04:10:23.313225+0000","last_unstale":"2024-03-29T04:10:25.297891+0000","last_undegraded":"2024-03-29T04:10:25.297891+0000","last_fullsized":"2024-03-29T04:10:25.297891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303498+0000","last_change":"2024-03-29T04:10:23.314124+0000","last_active":"2024-03-29T04:10:24.303498+0000","last_peered":"2024-03-29T04:10:24.303498+0000","last_clean":"2024-03-29T04:10:24.303498+0000","last_became_active":"2024-03-29T04:10:23.313434+0000","last_became_peered":"2024-03-29T04:10:23.313434+0000","last_unstale":"2024-03-29T04:10:24.303498+0000","last_undegraded":"2024-03-29T04:10:24.303498+0000","last_fullsized":"2024-03-29T04:10:24.303498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:36.507676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362465+0000","last_change":"2024-03-29T04:10:23.315461+0000","last_active":"2024-03-29T04:10:25.362465+0000","last_peered":"2024-03-29T04:10:25.362465+0000","last_clean":"2024-03-29T04:10:25.362465+0000","last_became_active":"2024-03-29T04:10:23.315163+0000","last_became_peered":"2024-03-29T04:10:23.315163+0000","last_unstale":"2024-03-29T04:10:25.362465+0000","last_undegraded":"2024-03-29T04:10:25.362465+0000","last_fullsized":"2024-03-29T04:10:25.362465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298321+0000","last_change":"2024-03-29T04:10:23.320116+0000","last_active":"2024-03-29T04:10:25.298321+0000","last_peered":"2024-03-29T04:10:25.298321+0000","last_clean":"2024-03-29T04:10:25.298321+0000","last_became_active":"2024-03-29T04:10:23.319801+0000","last_became_peered":"2024-03-29T04:10:23.319801+0000","last_unstale":"2024-03-29T04:10:25.298321+0000","last_undegraded":"2024-03-29T04:10:25.298321+0000","last_fullsized":"2024-03-29T04:10:25.298321+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.671436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298362+0000","last_change":"2024-03-29T04:10:23.315661+0000","last_active":"2024-03-29T04:10:25.298362+0000","last_peered":"2024-03-29T04:10:25.298362+0000","last_clean":"2024-03-29T04:10:25.298362+0000","last_became_active":"2024-03-29T04:10:23.315431+0000","last_became_peered":"2024-03-29T04:10:23.315431+0000","last_unstale":"2024-03-29T04:10:25.298362+0000","last_undegraded":"2024-03-29T04:10:25.298362+0000","last_fullsized":"2024-03-29T04:10:25.298362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:29.838326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362357+0000","last_change":"2024-03-29T04:10:23.309450+0000","last_active":"2024-03-29T04:10:25.362357+0000","last_peered":"2024-03-29T04:10:25.362357+0000","last_clean":"2024-03-29T04:10:25.362357+0000","last_became_active":"2024-03-29T04:10:23.309186+0000","last_became_peered":"2024-03-29T04:10:23.309186+0000","last_unstale":"2024-03-29T04:10:25.362357+0000","last_undegraded":"2024-03-29T04:10:25.362357+0000","last_fullsized":"2024-03-29T04:10:25.362357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298103+0000","last_change":"2024-03-29T04:10:23.319061+0000","last_active":"2024-03-29T04:10:25.298103+0000","last_peered":"2024-03-29T04:10:25.298103+0000","last_clean":"2024-03-29T04:10:25.298103+0000","last_became_active":"2024-03-29T04:10:23.318803+0000","last_became_peered":"2024-03-29T04:10:23.318803+0000","last_unstale":"2024-03-29T04:10:25.298103+0000","last_undegraded":"2024-03-29T04:10:25.298103+0000","last_fullsized":"2024-03-29T04:10:25.298103+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306790+0000","last_change":"2024-03-29T04:10:23.307357+0000","last_active":"2024-03-29T04:10:24.306790+0000","last_peered":"2024-03-29T04:10:24.306790+0000","last_clean":"2024-03-29T04:10:24.306790+0000","last_became_active":"2024-03-29T04:10:23.307035+0000","last_became_peered":"2024-03-29T04:10:23.307035+0000","last_unstale":"2024-03-29T04:10:24.306790+0000","last_undegraded":"2024-03-29T04:10:24.306790+0000","last_fullsized":"2024-03-29T04:10:24.306790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.297893+0000","last_change":"2024-03-29T04:10:23.310477+0000","last_active":"2024-03-29T04:10:25.297893+0000","last_peered":"2024-03-29T04:10:25.297893+0000","last_clean":"2024-03-29T04:10:25.297893+0000","last_became_active":"2024-03-29T04:10:23.309934+0000","last_became_peered":"2024-03-29T04:10:23.309934+0000","last_unstale":"2024-03-29T04:10:25.297893+0000","last_undegraded":"2024-03-29T04:10:25.297893+0000","last_fullsized":"2024-03-29T04:10:25.297893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362758+0000","last_change":"2024-03-29T04:10:23.311958+0000","last_active":"2024-03-29T04:10:25.362758+0000","last_peered":"2024-03-29T04:10:25.362758+0000","last_clean":"2024-03-29T04:10:25.362758+0000","last_became_active":"2024-03-29T04:10:23.311700+0000","last_became_peered":"2024-03-29T04:10:23.311700+0000","last_unstale":"2024-03-29T04:10:25.362758+0000","last_undegraded":"2024-03-29T04:10:25.362758+0000","last_fullsized":"2024-03-29T04:10:25.362758+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.363090+0000","last_change":"2024-03-29T04:10:23.313386+0000","last_active":"2024-03-29T04:10:25.363090+0000","last_peered":"2024-03-29T04:10:25.363090+0000","last_clean":"2024-03-29T04:10:25.363090+0000","last_became_active":"2024-03-29T04:10:23.313083+0000","last_became_peered":"2024-03-29T04:10:23.313083+0000","last_unstale":"2024-03-29T04:10:25.363090+0000","last_undegraded":"2024-03-29T04:10:25.363090+0000","last_fullsized":"2024-03-29T04:10:25.363090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:39.467534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307024+0000","last_change":"2024-03-29T04:10:23.300122+0000","last_active":"2024-03-29T04:10:24.307024+0000","last_peered":"2024-03-29T04:10:24.307024+0000","last_clean":"2024-03-29T04:10:24.307024+0000","last_became_active":"2024-03-29T04:10:23.299864+0000","last_became_peered":"2024-03-29T04:10:23.299864+0000","last_unstale":"2024-03-29T04:10:24.307024+0000","last_undegraded":"2024-03-29T04:10:24.307024+0000","last_fullsized":"2024-03-29T04:10:24.307024+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.135609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295814+0000","last_change":"2024-03-29T04:10:23.316162+0000","last_active":"2024-03-29T04:10:25.295814+0000","last_peered":"2024-03-29T04:10:25.295814+0000","last_clean":"2024-03-29T04:10:25.295814+0000","last_became_active":"2024-03-29T04:10:23.315894+0000","last_became_peered":"2024-03-29T04:10:23.315894+0000","last_unstale":"2024-03-29T04:10:25.295814+0000","last_undegraded":"2024-03-29T04:10:25.295814+0000","last_fullsized":"2024-03-29T04:10:25.295814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:21.815828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307313+0000","last_change":"2024-03-29T04:10:23.297888+0000","last_active":"2024-03-29T04:10:24.307313+0000","last_peered":"2024-03-29T04:10:24.307313+0000","last_clean":"2024-03-29T04:10:24.307313+0000","last_became_active":"2024-03-29T04:10:23.297611+0000","last_became_peered":"2024-03-29T04:10:23.297611+0000","last_unstale":"2024-03-29T04:10:24.307313+0000","last_undegraded":"2024-03-29T04:10:24.307313+0000","last_fullsized":"2024-03-29T04:10:24.307313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:18.112119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298091+0000","last_change":"2024-03-29T04:10:23.315440+0000","last_active":"2024-03-29T04:10:25.298091+0000","last_peered":"2024-03-29T04:10:25.298091+0000","last_clean":"2024-03-29T04:10:25.298091+0000","last_became_active":"2024-03-29T04:10:23.314317+0000","last_became_peered":"2024-03-29T04:10:23.314317+0000","last_unstale":"2024-03-29T04:10:25.298091+0000","last_undegraded":"2024-03-29T04:10:25.298091+0000","last_fullsized":"2024-03-29T04:10:25.298091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307246+0000","last_change":"2024-03-29T04:10:23.305362+0000","last_active":"2024-03-29T04:10:24.307246+0000","last_peered":"2024-03-29T04:10:24.307246+0000","last_clean":"2024-03-29T04:10:24.307246+0000","last_became_active":"2024-03-29T04:10:23.304734+0000","last_became_peered":"2024-03-29T04:10:23.304734+0000","last_unstale":"2024-03-29T04:10:24.307246+0000","last_undegraded":"2024-03-29T04:10:24.307246+0000","last_fullsized":"2024-03-29T04:10:24.307246+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362666+0000","last_change":"2024-03-29T04:10:23.315012+0000","last_active":"2024-03-29T04:10:25.362666+0000","last_peered":"2024-03-29T04:10:25.362666+0000","last_clean":"2024-03-29T04:10:25.362666+0000","last_became_active":"2024-03-29T04:10:23.312553+0000","last_became_peered":"2024-03-29T04:10:23.312553+0000","last_unstale":"2024-03-29T04:10:25.362666+0000","last_undegraded":"2024-03-29T04:10:25.362666+0000","last_fullsized":"2024-03-29T04:10:25.362666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295666+0000","last_change":"2024-03-29T04:10:23.318761+0000","last_active":"2024-03-29T04:10:25.295666+0000","last_peered":"2024-03-29T04:10:25.295666+0000","last_clean":"2024-03-29T04:10:25.295666+0000","last_became_active":"2024-03-29T04:10:23.318414+0000","last_became_peered":"2024-03-29T04:10:23.318414+0000","last_unstale":"2024-03-29T04:10:25.295666+0000","last_undegraded":"2024-03-29T04:10:25.295666+0000","last_fullsized":"2024-03-29T04:10:25.295666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:03:03.612190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307294+0000","last_change":"2024-03-29T04:10:23.312068+0000","last_active":"2024-03-29T04:10:24.307294+0000","last_peered":"2024-03-29T04:10:24.307294+0000","last_clean":"2024-03-29T04:10:24.307294+0000","last_became_active":"2024-03-29T04:10:23.311776+0000","last_became_peered":"2024-03-29T04:10:23.311776+0000","last_unstale":"2024-03-29T04:10:24.307294+0000","last_undegraded":"2024-03-29T04:10:24.307294+0000","last_fullsized":"2024-03-29T04:10:24.307294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:32.605320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487665+0000","last_change":"2024-03-29T04:10:23.318246+0000","last_active":"2024-03-29T04:10:24.487665+0000","last_peered":"2024-03-29T04:10:24.487665+0000","last_clean":"2024-03-29T04:10:24.487665+0000","last_became_active":"2024-03-29T04:10:23.317989+0000","last_became_peered":"2024-03-29T04:10:23.317989+0000","last_unstale":"2024-03-29T04:10:24.487665+0000","last_undegraded":"2024-03-29T04:10:24.487665+0000","last_fullsized":"2024-03-29T04:10:24.487665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307089+0000","last_change":"2024-03-29T04:10:23.308341+0000","last_active":"2024-03-29T04:10:24.307089+0000","last_peered":"2024-03-29T04:10:24.307089+0000","last_clean":"2024-03-29T04:10:24.307089+0000","last_became_active":"2024-03-29T04:10:23.307789+0000","last_became_peered":"2024-03-29T04:10:23.307789+0000","last_unstale":"2024-03-29T04:10:24.307089+0000","last_undegraded":"2024-03-29T04:10:24.307089+0000","last_fullsized":"2024-03-29T04:10:24.307089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:02.652851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298282+0000","last_change":"2024-03-29T04:10:23.316541+0000","last_active":"2024-03-29T04:10:25.298282+0000","last_peered":"2024-03-29T04:10:25.298282+0000","last_clean":"2024-03-29T04:10:25.298282+0000","last_became_active":"2024-03-29T04:10:23.316273+0000","last_became_peered":"2024-03-29T04:10:23.316273+0000","last_unstale":"2024-03-29T04:10:25.298282+0000","last_undegraded":"2024-03-29T04:10:25.298282+0000","last_fullsized":"2024-03-29T04:10:25.298282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:52.502645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306732+0000","last_change":"2024-03-29T04:10:23.307876+0000","last_active":"2024-03-29T04:10:24.306732+0000","last_peered":"2024-03-29T04:10:24.306732+0000","last_clean":"2024-03-29T04:10:24.306732+0000","last_became_active":"2024-03-29T04:10:23.307565+0000","last_became_peered":"2024-03-29T04:10:23.307565+0000","last_unstale":"2024-03-29T04:10:24.306732+0000","last_undegraded":"2024-03-29T04:10:24.306732+0000","last_fullsized":"2024-03-29T04:10:24.306732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:48.941110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306717+0000","last_change":"2024-03-29T04:10:23.306970+0000","last_active":"2024-03-29T04:10:24.306717+0000","last_peered":"2024-03-29T04:10:24.306717+0000","last_clean":"2024-03-29T04:10:24.306717+0000","last_became_active":"2024-03-29T04:10:23.306708+0000","last_became_peered":"2024-03-29T04:10:23.306708+0000","last_unstale":"2024-03-29T04:10:24.306717+0000","last_undegraded":"2024-03-29T04:10:24.306717+0000","last_fullsized":"2024-03-29T04:10:24.306717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:23.053923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362682+0000","last_change":"2024-03-29T04:10:23.314775+0000","last_active":"2024-03-29T04:10:25.362682+0000","last_peered":"2024-03-29T04:10:25.362682+0000","last_clean":"2024-03-29T04:10:25.362682+0000","last_became_active":"2024-03-29T04:10:23.312160+0000","last_became_peered":"2024-03-29T04:10:23.312160+0000","last_unstale":"2024-03-29T04:10:25.362682+0000","last_undegraded":"2024-03-29T04:10:25.362682+0000","last_fullsized":"2024-03-29T04:10:25.362682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487555+0000","last_change":"2024-03-29T04:10:23.304939+0000","last_active":"2024-03-29T04:10:24.487555+0000","last_peered":"2024-03-29T04:10:24.487555+0000","last_clean":"2024-03-29T04:10:24.487555+0000","last_became_active":"2024-03-29T04:10:23.304682+0000","last_became_peered":"2024-03-29T04:10:23.304682+0000","last_unstale":"2024-03-29T04:10:24.487555+0000","last_undegraded":"2024-03-29T04:10:24.487555+0000","last_fullsized":"2024-03-29T04:10:24.487555+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:02:58.170222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362886+0000","last_change":"2024-03-29T04:10:23.318116+0000","last_active":"2024-03-29T04:10:25.362886+0000","last_peered":"2024-03-29T04:10:25.362886+0000","last_clean":"2024-03-29T04:10:25.362886+0000","last_became_active":"2024-03-29T04:10:23.317859+0000","last_became_peered":"2024-03-29T04:10:23.317859+0000","last_unstale":"2024-03-29T04:10:25.362886+0000","last_undegraded":"2024-03-29T04:10:25.362886+0000","last_fullsized":"2024-03-29T04:10:25.362886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307305+0000","last_change":"2024-03-29T04:10:22.297429+0000","last_active":"2024-03-29T04:10:24.307305+0000","last_peered":"2024-03-29T04:10:24.307305+0000","last_clean":"2024-03-29T04:10:24.307305+0000","last_became_active":"2024-03-29T04:10:22.297077+0000","last_became_peered":"2024-03-29T04:10:22.297077+0000","last_unstale":"2024-03-29T04:10:24.307305+0000","last_undegraded":"2024-03-29T04:10:24.307305+0000","last_fullsized":"2024-03-29T04:10:24.307305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:10.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.488013+0000","last_change":"2024-03-29T04:10:23.316949+0000","last_active":"2024-03-29T04:10:24.488013+0000","last_peered":"2024-03-29T04:10:24.488013+0000","last_clean":"2024-03-29T04:10:24.488013+0000","last_became_active":"2024-03-29T04:10:23.316665+0000","last_became_peered":"2024-03-29T04:10:23.316665+0000","last_unstale":"2024-03-29T04:10:24.488013+0000","last_undegraded":"2024-03-29T04:10:24.488013+0000","last_fullsized":"2024-03-29T04:10:24.488013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362567+0000","last_change":"2024-03-29T04:10:22.307974+0000","last_active":"2024-03-29T04:10:25.362567+0000","last_peered":"2024-03-29T04:10:25.362567+0000","last_clean":"2024-03-29T04:10:25.362567+0000","last_became_active":"2024-03-29T04:10:22.306547+0000","last_became_peered":"2024-03-29T04:10:22.306547+0000","last_unstale":"2024-03-29T04:10:25.362567+0000","last_undegraded":"2024-03-29T04:10:25.362567+0000","last_fullsized":"2024-03-29T04:10:25.362567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307596+0000","last_change":"2024-03-29T04:10:23.302975+0000","last_active":"2024-03-29T04:10:24.307596+0000","last_peered":"2024-03-29T04:10:24.307596+0000","last_clean":"2024-03-29T04:10:24.307596+0000","last_became_active":"2024-03-29T04:10:23.302699+0000","last_became_peered":"2024-03-29T04:10:23.302699+0000","last_unstale":"2024-03-29T04:10:24.307596+0000","last_undegraded":"2024-03-29T04:10:24.307596+0000","last_fullsized":"2024-03-29T04:10:24.307596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487992+0000","last_change":"2024-03-29T04:10:22.301842+0000","last_active":"2024-03-29T04:10:24.487992+0000","last_peered":"2024-03-29T04:10:24.487992+0000","last_clean":"2024-03-29T04:10:24.487992+0000","last_became_active":"2024-03-29T04:10:22.301559+0000","last_became_peered":"2024-03-29T04:10:22.301559+0000","last_unstale":"2024-03-29T04:10:24.487992+0000","last_undegraded":"2024-03-29T04:10:24.487992+0000","last_fullsized":"2024-03-29T04:10:24.487992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.304037+0000","last_change":"2024-03-29T04:10:23.308697+0000","last_active":"2024-03-29T04:10:24.304037+0000","last_peered":"2024-03-29T04:10:24.304037+0000","last_clean":"2024-03-29T04:10:24.304037+0000","last_became_active":"2024-03-29T04:10:23.308430+0000","last_became_peered":"2024-03-29T04:10:23.308430+0000","last_unstale":"2024-03-29T04:10:24.304037+0000","last_undegraded":"2024-03-29T04:10:24.304037+0000","last_fullsized":"2024-03-29T04:10:24.304037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:06:07.478344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307158+0000","last_change":"2024-03-29T04:10:22.306797+0000","last_active":"2024-03-29T04:10:24.307158+0000","last_peered":"2024-03-29T04:10:24.307158+0000","last_clean":"2024-03-29T04:10:24.307158+0000","last_became_active":"2024-03-29T04:10:22.306504+0000","last_became_peered":"2024-03-29T04:10:22.306504+0000","last_unstale":"2024-03-29T04:10:24.307158+0000","last_undegraded":"2024-03-29T04:10:24.307158+0000","last_fullsized":"2024-03-29T04:10:24.307158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362273+0000","last_change":"2024-03-29T04:10:23.314380+0000","last_active":"2024-03-29T04:10:25.362273+0000","last_peered":"2024-03-29T04:10:25.362273+0000","last_clean":"2024-03-29T04:10:25.362273+0000","last_became_active":"2024-03-29T04:10:23.314129+0000","last_became_peered":"2024-03-29T04:10:23.314129+0000","last_unstale":"2024-03-29T04:10:25.362273+0000","last_undegraded":"2024-03-29T04:10:25.362273+0000","last_fullsized":"2024-03-29T04:10:25.362273+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298236+0000","last_change":"2024-03-29T04:10:22.310915+0000","last_active":"2024-03-29T04:10:25.298236+0000","last_peered":"2024-03-29T04:10:25.298236+0000","last_clean":"2024-03-29T04:10:25.298236+0000","last_became_active":"2024-03-29T04:10:22.310630+0000","last_became_peered":"2024-03-29T04:10:22.310630+0000","last_unstale":"2024-03-29T04:10:25.298236+0000","last_undegraded":"2024-03-29T04:10:25.298236+0000","last_fullsized":"2024-03-29T04:10:25.298236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298078+0000","last_change":"2024-03-29T04:10:23.314880+0000","last_active":"2024-03-29T04:10:25.298078+0000","last_peered":"2024-03-29T04:10:25.298078+0000","last_clean":"2024-03-29T04:10:25.298078+0000","last_became_active":"2024-03-29T04:10:23.313852+0000","last_became_peered":"2024-03-29T04:10:23.313852+0000","last_unstale":"2024-03-29T04:10:25.298078+0000","last_undegraded":"2024-03-29T04:10:25.298078+0000","last_fullsized":"2024-03-29T04:10:25.298078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:39:36.507676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362637+0000","last_change":"2024-03-29T04:10:22.304262+0000","last_active":"2024-03-29T04:10:25.362637+0000","last_peered":"2024-03-29T04:10:25.362637+0000","last_clean":"2024-03-29T04:10:25.362637+0000","last_became_active":"2024-03-29T04:10:22.303999+0000","last_became_peered":"2024-03-29T04:10:22.303999+0000","last_unstale":"2024-03-29T04:10:25.362637+0000","last_undegraded":"2024-03-29T04:10:25.362637+0000","last_fullsized":"2024-03-29T04:10:25.362637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306957+0000","last_change":"2024-03-29T04:10:23.317491+0000","last_active":"2024-03-29T04:10:24.306957+0000","last_peered":"2024-03-29T04:10:24.306957+0000","last_clean":"2024-03-29T04:10:24.306957+0000","last_became_active":"2024-03-29T04:10:23.317191+0000","last_became_peered":"2024-03-29T04:10:23.317191+0000","last_unstale":"2024-03-29T04:10:24.306957+0000","last_undegraded":"2024-03-29T04:10:24.306957+0000","last_fullsized":"2024-03-29T04:10:24.306957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487512+0000","last_change":"2024-03-29T04:10:22.307417+0000","last_active":"2024-03-29T04:10:24.487512+0000","last_peered":"2024-03-29T04:10:24.487512+0000","last_clean":"2024-03-29T04:10:24.487512+0000","last_became_active":"2024-03-29T04:10:22.307123+0000","last_became_peered":"2024-03-29T04:10:22.307123+0000","last_unstale":"2024-03-29T04:10:24.487512+0000","last_undegraded":"2024-03-29T04:10:24.487512+0000","last_fullsized":"2024-03-29T04:10:24.487512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303753+0000","last_change":"2024-03-29T04:10:23.308517+0000","last_active":"2024-03-29T04:10:24.303753+0000","last_peered":"2024-03-29T04:10:24.303753+0000","last_clean":"2024-03-29T04:10:24.303753+0000","last_became_active":"2024-03-29T04:10:23.308180+0000","last_became_peered":"2024-03-29T04:10:23.308180+0000","last_unstale":"2024-03-29T04:10:24.303753+0000","last_undegraded":"2024-03-29T04:10:24.303753+0000","last_fullsized":"2024-03-29T04:10:24.303753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:48:24.671436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306453+0000","last_change":"2024-03-29T04:10:22.294851+0000","last_active":"2024-03-29T04:10:24.306453+0000","last_peered":"2024-03-29T04:10:24.306453+0000","last_clean":"2024-03-29T04:10:24.306453+0000","last_became_active":"2024-03-29T04:10:22.294580+0000","last_became_peered":"2024-03-29T04:10:22.294580+0000","last_unstale":"2024-03-29T04:10:24.306453+0000","last_undegraded":"2024-03-29T04:10:24.306453+0000","last_fullsized":"2024-03-29T04:10:24.306453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:47:47.938803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295897+0000","last_change":"2024-03-29T04:10:23.313712+0000","last_active":"2024-03-29T04:10:25.295897+0000","last_peered":"2024-03-29T04:10:25.295897+0000","last_clean":"2024-03-29T04:10:25.295897+0000","last_became_active":"2024-03-29T04:10:23.313467+0000","last_became_peered":"2024-03-29T04:10:23.313467+0000","last_unstale":"2024-03-29T04:10:25.295897+0000","last_undegraded":"2024-03-29T04:10:25.295897+0000","last_fullsized":"2024-03-29T04:10:25.295897+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:11.192589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.304114+0000","last_change":"2024-03-29T04:10:22.305312+0000","last_active":"2024-03-29T04:10:24.304114+0000","last_peered":"2024-03-29T04:10:24.304114+0000","last_clean":"2024-03-29T04:10:24.304114+0000","last_became_active":"2024-03-29T04:10:22.304984+0000","last_became_peered":"2024-03-29T04:10:22.304984+0000","last_unstale":"2024-03-29T04:10:24.304114+0000","last_undegraded":"2024-03-29T04:10:24.304114+0000","last_fullsized":"2024-03-29T04:10:24.304114+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:35.565714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.363054+0000","last_change":"2024-03-29T04:10:23.304934+0000","last_active":"2024-03-29T04:10:25.363054+0000","last_peered":"2024-03-29T04:10:25.363054+0000","last_clean":"2024-03-29T04:10:25.363054+0000","last_became_active":"2024-03-29T04:10:23.304669+0000","last_became_peered":"2024-03-29T04:10:23.304669+0000","last_unstale":"2024-03-29T04:10:25.363054+0000","last_undegraded":"2024-03-29T04:10:25.363054+0000","last_fullsized":"2024-03-29T04:10:25.363054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:59:47.596318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298461+0000","last_change":"2024-03-29T04:10:22.308050+0000","last_active":"2024-03-29T04:10:25.298461+0000","last_peered":"2024-03-29T04:10:25.298461+0000","last_clean":"2024-03-29T04:10:25.298461+0000","last_became_active":"2024-03-29T04:10:22.307753+0000","last_became_peered":"2024-03-29T04:10:22.307753+0000","last_unstale":"2024-03-29T04:10:25.298461+0000","last_undegraded":"2024-03-29T04:10:25.298461+0000","last_fullsized":"2024-03-29T04:10:25.298461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:09:10.167872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362567+0000","last_change":"2024-03-29T04:10:23.317220+0000","last_active":"2024-03-29T04:10:25.362567+0000","last_peered":"2024-03-29T04:10:25.362567+0000","last_clean":"2024-03-29T04:10:25.362567+0000","last_became_active":"2024-03-29T04:10:23.316962+0000","last_became_peered":"2024-03-29T04:10:23.316962+0000","last_unstale":"2024-03-29T04:10:25.362567+0000","last_undegraded":"2024-03-29T04:10:25.362567+0000","last_fullsized":"2024-03-29T04:10:25.362567+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298501+0000","last_change":"2024-03-29T04:10:22.308995+0000","last_active":"2024-03-29T04:10:25.298501+0000","last_peered":"2024-03-29T04:10:25.298501+0000","last_clean":"2024-03-29T04:10:25.298501+0000","last_became_active":"2024-03-29T04:10:22.308679+0000","last_became_peered":"2024-03-29T04:10:22.308679+0000","last_unstale":"2024-03-29T04:10:25.298501+0000","last_undegraded":"2024-03-29T04:10:25.298501+0000","last_fullsized":"2024-03-29T04:10:25.298501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:07.976624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307252+0000","last_change":"2024-03-29T04:10:23.304261+0000","last_active":"2024-03-29T04:10:24.307252+0000","last_peered":"2024-03-29T04:10:24.307252+0000","last_clean":"2024-03-29T04:10:24.307252+0000","last_became_active":"2024-03-29T04:10:23.303966+0000","last_became_peered":"2024-03-29T04:10:23.303966+0000","last_unstale":"2024-03-29T04:10:24.307252+0000","last_undegraded":"2024-03-29T04:10:24.307252+0000","last_fullsized":"2024-03-29T04:10:24.307252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303935+0000","last_change":"2024-03-29T04:10:22.306679+0000","last_active":"2024-03-29T04:10:24.303935+0000","last_peered":"2024-03-29T04:10:24.303935+0000","last_clean":"2024-03-29T04:10:24.303935+0000","last_became_active":"2024-03-29T04:10:22.306411+0000","last_became_peered":"2024-03-29T04:10:22.306411+0000","last_unstale":"2024-03-29T04:10:24.303935+0000","last_undegraded":"2024-03-29T04:10:24.303935+0000","last_fullsized":"2024-03-29T04:10:24.303935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:22.259732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362274+0000","last_change":"2024-03-29T04:10:23.316218+0000","last_active":"2024-03-29T04:10:25.362274+0000","last_peered":"2024-03-29T04:10:25.362274+0000","last_clean":"2024-03-29T04:10:25.362274+0000","last_became_active":"2024-03-29T04:10:23.315899+0000","last_became_peered":"2024-03-29T04:10:23.315899+0000","last_unstale":"2024-03-29T04:10:25.362274+0000","last_undegraded":"2024-03-29T04:10:25.362274+0000","last_fullsized":"2024-03-29T04:10:25.362274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:36.093598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303883+0000","last_change":"2024-03-29T04:10:22.310885+0000","last_active":"2024-03-29T04:10:24.303883+0000","last_peered":"2024-03-29T04:10:24.303883+0000","last_clean":"2024-03-29T04:10:24.303883+0000","last_became_active":"2024-03-29T04:10:22.310589+0000","last_became_peered":"2024-03-29T04:10:22.310589+0000","last_unstale":"2024-03-29T04:10:24.303883+0000","last_undegraded":"2024-03-29T04:10:24.303883+0000","last_fullsized":"2024-03-29T04:10:24.303883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:00.859522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303607+0000","last_change":"2024-03-29T04:10:23.312331+0000","last_active":"2024-03-29T04:10:24.303607+0000","last_peered":"2024-03-29T04:10:24.303607+0000","last_clean":"2024-03-29T04:10:24.303607+0000","last_became_active":"2024-03-29T04:10:23.311961+0000","last_became_peered":"2024-03-29T04:10:23.311961+0000","last_unstale":"2024-03-29T04:10:24.303607+0000","last_undegraded":"2024-03-29T04:10:24.303607+0000","last_fullsized":"2024-03-29T04:10:24.303607+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:00:52.502645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487789+0000","last_change":"2024-03-29T04:10:22.308119+0000","last_active":"2024-03-29T04:10:24.487789+0000","last_peered":"2024-03-29T04:10:24.487789+0000","last_clean":"2024-03-29T04:10:24.487789+0000","last_became_active":"2024-03-29T04:10:22.307849+0000","last_became_peered":"2024-03-29T04:10:22.307849+0000","last_unstale":"2024-03-29T04:10:24.487789+0000","last_undegraded":"2024-03-29T04:10:24.487789+0000","last_fullsized":"2024-03-29T04:10:24.487789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362795+0000","last_change":"2024-03-29T04:10:23.310446+0000","last_active":"2024-03-29T04:10:25.362795+0000","last_peered":"2024-03-29T04:10:25.362795+0000","last_clean":"2024-03-29T04:10:25.362795+0000","last_became_active":"2024-03-29T04:10:23.310183+0000","last_became_peered":"2024-03-29T04:10:23.310183+0000","last_unstale":"2024-03-29T04:10:25.362795+0000","last_undegraded":"2024-03-29T04:10:25.362795+0000","last_fullsized":"2024-03-29T04:10:25.362795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306550+0000","last_change":"2024-03-29T04:10:22.297317+0000","last_active":"2024-03-29T04:10:24.306550+0000","last_peered":"2024-03-29T04:10:24.306550+0000","last_clean":"2024-03-29T04:10:24.306550+0000","last_became_active":"2024-03-29T04:10:22.297044+0000","last_became_peered":"2024-03-29T04:10:22.297044+0000","last_unstale":"2024-03-29T04:10:24.306550+0000","last_undegraded":"2024-03-29T04:10:24.306550+0000","last_fullsized":"2024-03-29T04:10:24.306550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:56.164692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303789+0000","last_change":"2024-03-29T04:10:23.312644+0000","last_active":"2024-03-29T04:10:24.303789+0000","last_peered":"2024-03-29T04:10:24.303789+0000","last_clean":"2024-03-29T04:10:24.303789+0000","last_became_active":"2024-03-29T04:10:23.312370+0000","last_became_peered":"2024-03-29T04:10:23.312370+0000","last_unstale":"2024-03-29T04:10:24.303789+0000","last_undegraded":"2024-03-29T04:10:24.303789+0000","last_fullsized":"2024-03-29T04:10:24.303789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:29.838326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362533+0000","last_change":"2024-03-29T04:10:22.304877+0000","last_active":"2024-03-29T04:10:25.362533+0000","last_peered":"2024-03-29T04:10:25.362533+0000","last_clean":"2024-03-29T04:10:25.362533+0000","last_became_active":"2024-03-29T04:10:22.304573+0000","last_became_peered":"2024-03-29T04:10:22.304573+0000","last_unstale":"2024-03-29T04:10:25.362533+0000","last_undegraded":"2024-03-29T04:10:25.362533+0000","last_fullsized":"2024-03-29T04:10:25.362533+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303914+0000","last_change":"2024-03-29T04:10:23.309896+0000","last_active":"2024-03-29T04:10:24.303914+0000","last_peered":"2024-03-29T04:10:24.303914+0000","last_clean":"2024-03-29T04:10:24.303914+0000","last_became_active":"2024-03-29T04:10:23.308057+0000","last_became_peered":"2024-03-29T04:10:23.308057+0000","last_unstale":"2024-03-29T04:10:24.303914+0000","last_undegraded":"2024-03-29T04:10:24.303914+0000","last_fullsized":"2024-03-29T04:10:24.303914+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:28:10.989656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298428+0000","last_change":"2024-03-29T04:10:22.307155+0000","last_active":"2024-03-29T04:10:25.298428+0000","last_peered":"2024-03-29T04:10:25.298428+0000","last_clean":"2024-03-29T04:10:25.298428+0000","last_became_active":"2024-03-29T04:10:22.306817+0000","last_became_peered":"2024-03-29T04:10:22.306817+0000","last_unstale":"2024-03-29T04:10:25.298428+0000","last_undegraded":"2024-03-29T04:10:25.298428+0000","last_fullsized":"2024-03-29T04:10:25.298428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:31.603013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298471+0000","last_change":"2024-03-29T04:10:23.315554+0000","last_active":"2024-03-29T04:10:25.298471+0000","last_peered":"2024-03-29T04:10:25.298471+0000","last_clean":"2024-03-29T04:10:25.298471+0000","last_became_active":"2024-03-29T04:10:23.314900+0000","last_became_peered":"2024-03-29T04:10:23.314900+0000","last_unstale":"2024-03-29T04:10:25.298471+0000","last_undegraded":"2024-03-29T04:10:25.298471+0000","last_fullsized":"2024-03-29T04:10:25.298471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:28:10.989656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307665+0000","last_change":"2024-03-29T04:10:22.307690+0000","last_active":"2024-03-29T04:10:24.307665+0000","last_peered":"2024-03-29T04:10:24.307665+0000","last_clean":"2024-03-29T04:10:24.307665+0000","last_became_active":"2024-03-29T04:10:22.307424+0000","last_became_peered":"2024-03-29T04:10:22.307424+0000","last_unstale":"2024-03-29T04:10:24.307665+0000","last_undegraded":"2024-03-29T04:10:24.307665+0000","last_fullsized":"2024-03-29T04:10:24.307665+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307557+0000","last_change":"2024-03-29T04:10:23.314568+0000","last_active":"2024-03-29T04:10:24.307557+0000","last_peered":"2024-03-29T04:10:24.307557+0000","last_clean":"2024-03-29T04:10:24.307557+0000","last_became_active":"2024-03-29T04:10:23.314307+0000","last_became_peered":"2024-03-29T04:10:23.314307+0000","last_unstale":"2024-03-29T04:10:24.307557+0000","last_undegraded":"2024-03-29T04:10:24.307557+0000","last_fullsized":"2024-03-29T04:10:24.307557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295816+0000","last_change":"2024-03-29T04:10:22.303744+0000","last_active":"2024-03-29T04:10:25.295816+0000","last_peered":"2024-03-29T04:10:25.295816+0000","last_clean":"2024-03-29T04:10:25.295816+0000","last_became_active":"2024-03-29T04:10:22.303477+0000","last_became_peered":"2024-03-29T04:10:22.303477+0000","last_unstale":"2024-03-29T04:10:25.295816+0000","last_undegraded":"2024-03-29T04:10:25.295816+0000","last_fullsized":"2024-03-29T04:10:25.295816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:03:07.972943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295928+0000","last_change":"2024-03-29T04:10:23.306990+0000","last_active":"2024-03-29T04:10:25.295928+0000","last_peered":"2024-03-29T04:10:25.295928+0000","last_clean":"2024-03-29T04:10:25.295928+0000","last_became_active":"2024-03-29T04:10:23.306635+0000","last_became_peered":"2024-03-29T04:10:23.306635+0000","last_unstale":"2024-03-29T04:10:25.295928+0000","last_undegraded":"2024-03-29T04:10:25.295928+0000","last_fullsized":"2024-03-29T04:10:25.295928+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:29.534483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487771+0000","last_change":"2024-03-29T04:10:22.300528+0000","last_active":"2024-03-29T04:10:24.487771+0000","last_peered":"2024-03-29T04:10:24.487771+0000","last_clean":"2024-03-29T04:10:24.487771+0000","last_became_active":"2024-03-29T04:10:22.299874+0000","last_became_peered":"2024-03-29T04:10:22.299874+0000","last_unstale":"2024-03-29T04:10:24.487771+0000","last_undegraded":"2024-03-29T04:10:24.487771+0000","last_fullsized":"2024-03-29T04:10:24.487771+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:01.650545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362909+0000","last_change":"2024-03-29T04:10:23.311796+0000","last_active":"2024-03-29T04:10:25.362909+0000","last_peered":"2024-03-29T04:10:25.362909+0000","last_clean":"2024-03-29T04:10:25.362909+0000","last_became_active":"2024-03-29T04:10:23.311351+0000","last_became_peered":"2024-03-29T04:10:23.311351+0000","last_unstale":"2024-03-29T04:10:25.362909+0000","last_undegraded":"2024-03-29T04:10:25.362909+0000","last_fullsized":"2024-03-29T04:10:25.362909+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:07:06.265360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295766+0000","last_change":"2024-03-29T04:10:22.308134+0000","last_active":"2024-03-29T04:10:25.295766+0000","last_peered":"2024-03-29T04:10:25.295766+0000","last_clean":"2024-03-29T04:10:25.295766+0000","last_became_active":"2024-03-29T04:10:22.307860+0000","last_became_peered":"2024-03-29T04:10:22.307860+0000","last_unstale":"2024-03-29T04:10:25.295766+0000","last_undegraded":"2024-03-29T04:10:25.295766+0000","last_fullsized":"2024-03-29T04:10:25.295766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:20:17.587318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362419+0000","last_change":"2024-03-29T04:10:23.309927+0000","last_active":"2024-03-29T04:10:25.362419+0000","last_peered":"2024-03-29T04:10:25.362419+0000","last_clean":"2024-03-29T04:10:25.362419+0000","last_became_active":"2024-03-29T04:10:23.309623+0000","last_became_peered":"2024-03-29T04:10:23.309623+0000","last_unstale":"2024-03-29T04:10:25.362419+0000","last_undegraded":"2024-03-29T04:10:25.362419+0000","last_fullsized":"2024-03-29T04:10:25.362419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:31:49.558260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295692+0000","last_change":"2024-03-29T04:10:22.297913+0000","last_active":"2024-03-29T04:10:25.295692+0000","last_peered":"2024-03-29T04:10:25.295692+0000","last_clean":"2024-03-29T04:10:25.295692+0000","last_became_active":"2024-03-29T04:10:22.296015+0000","last_became_peered":"2024-03-29T04:10:22.296015+0000","last_unstale":"2024-03-29T04:10:25.295692+0000","last_undegraded":"2024-03-29T04:10:25.295692+0000","last_fullsized":"2024-03-29T04:10:25.295692+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:13.154992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.297986+0000","last_change":"2024-03-29T04:10:23.315943+0000","last_active":"2024-03-29T04:10:25.297986+0000","last_peered":"2024-03-29T04:10:25.297986+0000","last_clean":"2024-03-29T04:10:25.297986+0000","last_became_active":"2024-03-29T04:10:23.311693+0000","last_became_peered":"2024-03-29T04:10:23.311693+0000","last_unstale":"2024-03-29T04:10:25.297986+0000","last_undegraded":"2024-03-29T04:10:25.297986+0000","last_fullsized":"2024-03-29T04:10:25.297986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:16:17.418204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362805+0000","last_change":"2024-03-29T04:10:22.305287+0000","last_active":"2024-03-29T04:10:25.362805+0000","last_peered":"2024-03-29T04:10:25.362805+0000","last_clean":"2024-03-29T04:10:25.362805+0000","last_became_active":"2024-03-29T04:10:22.304999+0000","last_became_peered":"2024-03-29T04:10:22.304999+0000","last_unstale":"2024-03-29T04:10:25.362805+0000","last_undegraded":"2024-03-29T04:10:25.362805+0000","last_fullsized":"2024-03-29T04:10:25.362805+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303910+0000","last_change":"2024-03-29T04:10:23.307638+0000","last_active":"2024-03-29T04:10:24.303910+0000","last_peered":"2024-03-29T04:10:24.303910+0000","last_clean":"2024-03-29T04:10:24.303910+0000","last_became_active":"2024-03-29T04:10:23.307381+0000","last_became_peered":"2024-03-29T04:10:23.307381+0000","last_unstale":"2024-03-29T04:10:24.303910+0000","last_undegraded":"2024-03-29T04:10:24.303910+0000","last_fullsized":"2024-03-29T04:10:24.303910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:41.931099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295618+0000","last_change":"2024-03-29T04:10:22.307254+0000","last_active":"2024-03-29T04:10:25.295618+0000","last_peered":"2024-03-29T04:10:25.295618+0000","last_clean":"2024-03-29T04:10:25.295618+0000","last_became_active":"2024-03-29T04:10:22.306976+0000","last_became_peered":"2024-03-29T04:10:22.306976+0000","last_unstale":"2024-03-29T04:10:25.295618+0000","last_undegraded":"2024-03-29T04:10:25.295618+0000","last_fullsized":"2024-03-29T04:10:25.295618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:54:11.970574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487677+0000","last_change":"2024-03-29T04:10:23.317609+0000","last_active":"2024-03-29T04:10:24.487677+0000","last_peered":"2024-03-29T04:10:24.487677+0000","last_clean":"2024-03-29T04:10:24.487677+0000","last_became_active":"2024-03-29T04:10:23.317364+0000","last_became_peered":"2024-03-29T04:10:23.317364+0000","last_unstale":"2024-03-29T04:10:24.487677+0000","last_undegraded":"2024-03-29T04:10:24.487677+0000","last_fullsized":"2024-03-29T04:10:24.487677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:26.593833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298565+0000","last_change":"2024-03-29T04:10:22.308920+0000","last_active":"2024-03-29T04:10:25.298565+0000","last_peered":"2024-03-29T04:10:25.298565+0000","last_clean":"2024-03-29T04:10:25.298565+0000","last_became_active":"2024-03-29T04:10:22.308648+0000","last_became_peered":"2024-03-29T04:10:22.308648+0000","last_unstale":"2024-03-29T04:10:25.298565+0000","last_undegraded":"2024-03-29T04:10:25.298565+0000","last_fullsized":"2024-03-29T04:10:25.298565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:38:17.109812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.363028+0000","last_change":"2024-03-29T04:10:23.305533+0000","last_active":"2024-03-29T04:10:25.363028+0000","last_peered":"2024-03-29T04:10:25.363028+0000","last_clean":"2024-03-29T04:10:25.363028+0000","last_became_active":"2024-03-29T04:10:23.305295+0000","last_became_peered":"2024-03-29T04:10:23.305295+0000","last_unstale":"2024-03-29T04:10:25.363028+0000","last_undegraded":"2024-03-29T04:10:25.363028+0000","last_fullsized":"2024-03-29T04:10:25.363028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:06:17.883608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.295883+0000","last_change":"2024-03-29T04:10:22.309267+0000","last_active":"2024-03-29T04:10:25.295883+0000","last_peered":"2024-03-29T04:10:25.295883+0000","last_clean":"2024-03-29T04:10:25.295883+0000","last_became_active":"2024-03-29T04:10:22.308996+0000","last_became_peered":"2024-03-29T04:10:22.308996+0000","last_unstale":"2024-03-29T04:10:25.295883+0000","last_undegraded":"2024-03-29T04:10:25.295883+0000","last_fullsized":"2024-03-29T04:10:25.295883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:27.354497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.304132+0000","last_change":"2024-03-29T04:10:23.313551+0000","last_active":"2024-03-29T04:10:24.304132+0000","last_peered":"2024-03-29T04:10:24.304132+0000","last_clean":"2024-03-29T04:10:24.304132+0000","last_became_active":"2024-03-29T04:10:23.313289+0000","last_became_peered":"2024-03-29T04:10:23.313289+0000","last_unstale":"2024-03-29T04:10:24.304132+0000","last_undegraded":"2024-03-29T04:10:24.304132+0000","last_fullsized":"2024-03-29T04:10:24.304132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:37.649554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362441+0000","last_change":"2024-03-29T04:10:22.308648+0000","last_active":"2024-03-29T04:10:25.362441+0000","last_peered":"2024-03-29T04:10:25.362441+0000","last_clean":"2024-03-29T04:10:25.362441+0000","last_became_active":"2024-03-29T04:10:22.308334+0000","last_became_peered":"2024-03-29T04:10:22.308334+0000","last_unstale":"2024-03-29T04:10:25.362441+0000","last_undegraded":"2024-03-29T04:10:25.362441+0000","last_fullsized":"2024-03-29T04:10:25.362441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T06:18:58.918922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362434+0000","last_change":"2024-03-29T04:10:23.319502+0000","last_active":"2024-03-29T04:10:25.362434+0000","last_peered":"2024-03-29T04:10:25.362434+0000","last_clean":"2024-03-29T04:10:25.362434+0000","last_became_active":"2024-03-29T04:10:23.319209+0000","last_became_peered":"2024-03-29T04:10:23.319209+0000","last_unstale":"2024-03-29T04:10:25.362434+0000","last_undegraded":"2024-03-29T04:10:25.362434+0000","last_fullsized":"2024-03-29T04:10:25.362434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:51:42.169170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362666+0000","last_change":"2024-03-29T04:10:22.307995+0000","last_active":"2024-03-29T04:10:25.362666+0000","last_peered":"2024-03-29T04:10:25.362666+0000","last_clean":"2024-03-29T04:10:25.362666+0000","last_became_active":"2024-03-29T04:10:22.307723+0000","last_became_peered":"2024-03-29T04:10:22.307723+0000","last_unstale":"2024-03-29T04:10:25.362666+0000","last_undegraded":"2024-03-29T04:10:25.362666+0000","last_fullsized":"2024-03-29T04:10:25.362666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:22.051616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.487518+0000","last_change":"2024-03-29T04:10:23.308586+0000","last_active":"2024-03-29T04:10:24.487518+0000","last_peered":"2024-03-29T04:10:24.487518+0000","last_clean":"2024-03-29T04:10:24.487518+0000","last_became_active":"2024-03-29T04:10:23.308278+0000","last_became_peered":"2024-03-29T04:10:23.308278+0000","last_unstale":"2024-03-29T04:10:24.487518+0000","last_undegraded":"2024-03-29T04:10:24.487518+0000","last_fullsized":"2024-03-29T04:10:24.487518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:23.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307081+0000","last_change":"2024-03-29T04:10:22.309876+0000","last_active":"2024-03-29T04:10:24.307081+0000","last_peered":"2024-03-29T04:10:24.307081+0000","last_clean":"2024-03-29T04:10:24.307081+0000","last_became_active":"2024-03-29T04:10:22.309473+0000","last_became_peered":"2024-03-29T04:10:22.309473+0000","last_unstale":"2024-03-29T04:10:24.307081+0000","last_undegraded":"2024-03-29T04:10:24.307081+0000","last_fullsized":"2024-03-29T04:10:24.307081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T05:20:20.813521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.307452+0000","last_change":"2024-03-29T04:10:23.305607+0000","last_active":"2024-03-29T04:10:24.307452+0000","last_peered":"2024-03-29T04:10:24.307452+0000","last_clean":"2024-03-29T04:10:24.307452+0000","last_became_active":"2024-03-29T04:10:23.305065+0000","last_became_peered":"2024-03-29T04:10:23.305065+0000","last_unstale":"2024-03-29T04:10:24.307452+0000","last_undegraded":"2024-03-29T04:10:24.307452+0000","last_fullsized":"2024-03-29T04:10:24.307452+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:59.899583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.306614+0000","last_change":"2024-03-29T04:10:22.302187+0000","last_active":"2024-03-29T04:10:24.306614+0000","last_peered":"2024-03-29T04:10:24.306614+0000","last_clean":"2024-03-29T04:10:24.306614+0000","last_became_active":"2024-03-29T04:10:22.301830+0000","last_became_peered":"2024-03-29T04:10:22.301830+0000","last_unstale":"2024-03-29T04:10:24.306614+0000","last_undegraded":"2024-03-29T04:10:24.306614+0000","last_fullsized":"2024-03-29T04:10:24.306614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362917+0000","last_change":"2024-03-29T04:10:23.303127+0000","last_active":"2024-03-29T04:10:25.362917+0000","last_peered":"2024-03-29T04:10:25.362917+0000","last_clean":"2024-03-29T04:10:25.362917+0000","last_became_active":"2024-03-29T04:10:23.302882+0000","last_became_peered":"2024-03-29T04:10:23.302882+0000","last_unstale":"2024-03-29T04:10:25.362917+0000","last_undegraded":"2024-03-29T04:10:25.362917+0000","last_fullsized":"2024-03-29T04:10:25.362917+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T07:18:04.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:24.303482+0000","last_change":"2024-03-29T04:10:22.304632+0000","last_active":"2024-03-29T04:10:24.303482+0000","last_peered":"2024-03-29T04:10:24.303482+0000","last_clean":"2024-03-29T04:10:24.303482+0000","last_became_active":"2024-03-29T04:10:22.304363+0000","last_became_peered":"2024-03-29T04:10:22.304363+0000","last_unstale":"2024-03-29T04:10:24.303482+0000","last_undegraded":"2024-03-29T04:10:24.303482+0000","last_fullsized":"2024-03-29T04:10:24.303482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:35.091291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.298397+0000","last_change":"2024-03-29T04:10:23.315767+0000","last_active":"2024-03-29T04:10:25.298397+0000","last_peered":"2024-03-29T04:10:25.298397+0000","last_clean":"2024-03-29T04:10:25.298397+0000","last_became_active":"2024-03-29T04:10:23.315186+0000","last_became_peered":"2024-03-29T04:10:23.315186+0000","last_unstale":"2024-03-29T04:10:25.298397+0000","last_undegraded":"2024-03-29T04:10:25.298397+0000","last_fullsized":"2024-03-29T04:10:25.298397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:22.281232+0000","last_clean_scrub_stamp":"2024-03-29T04:10:22.281232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:39:41.931099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362766+0000","last_change":"2024-03-29T04:10:22.310649+0000","last_active":"2024-03-29T04:10:25.362766+0000","last_peered":"2024-03-29T04:10:25.362766+0000","last_clean":"2024-03-29T04:10:25.362766+0000","last_became_active":"2024-03-29T04:10:22.310382+0000","last_became_peered":"2024-03-29T04:10:22.310382+0000","last_unstale":"2024-03-29T04:10:25.362766+0000","last_undegraded":"2024-03-29T04:10:25.362766+0000","last_fullsized":"2024-03-29T04:10:25.362766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:10:21.278925+0000","last_clean_scrub_stamp":"2024-03-29T04:10:21.278925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:09.133302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T04:10:25.362889+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:25.362889+0000","last_peered":"2024-03-29T04:10:25.362889+0000","last_clean":"2024-03-29T04:10:25.362889+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:25.362889+0000","last_undegraded":"2024-03-29T04:10:25.362889+0000","last_fullsized":"2024-03-29T04:10:25.362889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":789223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":197285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":197243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":787575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:10:31.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T04:10:31.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T04:10:31.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.461+0000 7faf495db640 1 Processor -- start 2024-03-29T04:10:31.441 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.461+0000 7faf495db640 1 -- start start 2024-03-29T04:10:31.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf495db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf440b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf495db640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faf440d3ee0 0x7faf440d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.453 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf495db640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf440d2e40 con 0x7faf4415e4f0 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf495db640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faf440d3a50 con 0x7faf440d3ee0 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf495db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf440d3120 con 0x7faf440b6520 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf42ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faf4415e4f0 0x7faf440b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39090/0 (socket says 172.21.15.181:39090) 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf42ffd640 1 -- 172.21.15.181:0/4150849018 learned_addr learned my addr 172.21.15.181:0/4150849018 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886568678 0 0) 0x7faf440d2e40 con 0x7faf4415e4f0 2024-03-29T04:10:31.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf437fe640 1 --2- 172.21.15.181:0/4150849018 >> v2:172.21.15.187:3300/0 conn(0x7faf440d3ee0 0x7faf440d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf30003520 con 0x7faf4415e4f0 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.462+0000 7faf427fc640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf440b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2648185551 0 0) 0x7faf30003520 con 0x7faf4415e4f0 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 msgr2=0x7faf440b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf440b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 >> v2:172.21.15.187:3300/0 conn(0x7faf440d3ee0 msgr2=0x7faf440d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 --2- 172.21.15.181:0/4150849018 >> v2:172.21.15.187:3300/0 conn(0x7faf440d3ee0 0x7faf440d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf440d3400 con 0x7faf4415e4f0 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7faf34001670 con 0x7faf4415e4f0 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 >> v1:172.21.15.181:6789/0 conn(0x7faf4415e4f0 legacy=0x7faf440b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf300039b0 0x7faf30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf30004800 0x7faf30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf440d3a50 con 0x7faf300039b0 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf440d3120 con 0x7faf30004800 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf3000b9c0 con 0x7faf440b6520 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf300039b0 0x7faf30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf427fc640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf30004800 0x7faf30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf437fe640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 -- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 msgr2=0x7faf300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf440b6520 0x7faf300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 -- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf30004800 msgr2=0x7faf30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf30004800 0x7faf30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.463+0000 7faf42ffd640 1 -- 172.21.15.181:0/4150849018 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf30003520 con 0x7faf300039b0 2024-03-29T04:10:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf42ffd640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf300039b0 0x7faf30003dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf34001050 tx=0x7faf30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3669b5ea52a65066 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34007d70 con 0x7faf300039b0 2024-03-29T04:10:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34030070 con 0x7faf300039b0 2024-03-29T04:10:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf41ffb640 1 -- 172.21.15.181:0/4150849018 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3402cd80 con 0x7faf300039b0 2024-03-29T04:10:31.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf495db640 1 -- 172.21.15.181:0/4150849018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf300039b0 msgr2=0x7faf30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf495db640 1 --2- 172.21.15.181:0/4150849018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf300039b0 0x7faf30003dd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf34001050 tx=0x7faf30005830 comp rx=0 tx=0).stop 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.464+0000 7faf437fe640 1 -- 172.21.15.181:0/4150849018 reap_dead start 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.465+0000 7faf495db640 1 -- 172.21.15.181:0/4150849018 shutdown_connections 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.465+0000 7faf495db640 1 -- 172.21.15.181:0/4150849018 >> 172.21.15.181:0/4150849018 conn(0x7faf440678f0 msgr2=0x7faf440d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.465+0000 7faf495db640 1 -- 172.21.15.181:0/4150849018 shutdown_connections 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.465+0000 7faf495db640 1 -- 172.21.15.181:0/4150849018 wait complete. 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.466+0000 7faf495db640 1 Processor -- start 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.466+0000 7faf495db640 1 -- start start 2024-03-29T04:10:31.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 0x7faf440ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 0x7faf441566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf440b9f20 con 0x7faf44159200 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf4415e8d0 con 0x7faf440cb460 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf440ba0a0 con 0x7faf300039b0 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39602/0 (socket says 172.21.15.181:39602) 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 -- 172.21.15.181:0/2828598326 learned_addr learned my addr 172.21.15.181:0/2828598326 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf42ffd640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 0x7faf440ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf427fc640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 0x7faf441566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 -- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 msgr2=0x7faf440ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 0x7faf440ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 -- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 msgr2=0x7faf441566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 0x7faf441566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 -- 172.21.15.181:0/2828598326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf44156c30 con 0x7faf44159200 2024-03-29T04:10:31.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf42ffd640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 0x7faf440ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf427fc640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 0x7faf441566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf437fe640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf440d1ef0 tx=0x7faf3c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c00fdb0 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3c01d070 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf44156ee0 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c015670 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.467+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf441574a0 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.468+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf44075890 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.469+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faf3c015810 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.469+0000 7faf3b7fe640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 0x7faf2406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.469+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7faf3c08c030 con 0x7faf44159200 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.469+0000 7faf42ffd640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 0x7faf2406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.470+0000 7faf42ffd640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 0x7faf2406ea20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf34034000 tx=0x7faf34007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.475+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faf3c010040 con 0x7faf44159200 2024-03-29T04:10:31.698 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.719+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faf440b99f0 con 0x7faf44159200 2024-03-29T04:10:31.702 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.723+0000 7faf3b7fe640 1 -- 172.21.15.181:0/2828598326 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7faf3c05a4c0 con 0x7faf44159200 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCsPgZm/aIPNxAAtuCXeNhwViJIHObDAGDzrw== 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stdout: caps mds = "allow" 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stdout: caps mon = "allow r" 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-29T04:10:31.703 INFO:teuthology.orchestra.run.smithi181.stderr:updated caps for client.0 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 msgr2=0x7faf2406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 0x7faf2406ea20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf34034000 tx=0x7faf34007000 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 msgr2=0x7faf44159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf440d1ef0 tx=0x7faf3c00f580 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 shutdown_connections 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf44159200 0x7faf44159680 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf440cb460 0x7faf441566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7faf2406c560 0x7faf2406ea20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 --2- 172.21.15.181:0/2828598326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf300039b0 0x7faf440ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 >> 172.21.15.181:0/2828598326 conn(0x7faf440678f0 msgr2=0x7faf4415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 shutdown_connections 2024-03-29T04:10:31.705 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.725+0000 7faf495db640 1 -- 172.21.15.181:0/2828598326 wait complete. 2024-03-29T04:10:31.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:10:31.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.838+0000 7fb1a52c1640 1 Processor -- start 2024-03-29T04:10:31.818 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.838+0000 7fb1a52c1640 1 -- start start 2024-03-29T04:10:31.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1a52c1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a015e950 0x7fb1a00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1a52c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1a52c1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a00d36f0 con 0x7fb1a00d2f00 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1a52c1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a00d3cb0 con 0x7fb1a015e950 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1a52c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a00d4d00 con 0x7fb1a00d48c0 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a015e950 0x7fb1a00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1a015e950 0x7fb1a00d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39612/0 (socket says 172.21.15.181:39612) 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 -- 172.21.15.181:0/3968772530 learned_addr learned my addr 172.21.15.181:0/3968772530 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb1967fc640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 -- 172.21.15.181:0/3968772530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 msgr2=0x7fb1a00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a00d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 -- 172.21.15.181:0/3968772530 >> v1:172.21.15.187:6789/0 conn(0x7fb1a00d2f00 legacy=0x7fb1a00cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.840+0000 7fb19effd640 1 -- 172.21.15.181:0/3968772530 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a00d39d0 con 0x7fb1a015e950 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.841+0000 7fb19effd640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a00d4380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb190009870 tx=0x7fb19002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e071f74532815a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.841+0000 7fb19e7fc640 1 -- 172.21.15.181:0/3968772530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19002f840 con 0x7fb1a015e950 2024-03-29T04:10:31.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.841+0000 7fb19e7fc640 1 -- 172.21.15.181:0/3968772530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19002f9e0 con 0x7fb1a015e950 2024-03-29T04:10:31.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.841+0000 7fb19e7fc640 1 -- 172.21.15.181:0/3968772530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19002fdb0 con 0x7fb1a015e950 2024-03-29T04:10:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 -- 172.21.15.181:0/3968772530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 msgr2=0x7fb1a00d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.824 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a00d4380 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb190009870 tx=0x7fb19002ba20 comp rx=0 tx=0).stop 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 -- 172.21.15.181:0/3968772530 shutdown_connections 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a00d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/3968772530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a00d4380 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 -- 172.21.15.181:0/3968772530 >> 172.21.15.181:0/3968772530 conn(0x7fb1a00676e0 msgr2=0x7fb1a0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 -- 172.21.15.181:0/3968772530 shutdown_connections 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.842+0000 7fb1a52c1640 1 -- 172.21.15.181:0/3968772530 wait complete. 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.843+0000 7fb1a52c1640 1 Processor -- start 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.843+0000 7fb1a52c1640 1 -- start start 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00d3390 con 0x7fb1a015e950 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00b7da0 con 0x7fb1a00d2f00 2024-03-29T04:10:31.825 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1a52c1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a015ed30 con 0x7fb1a00d48c0 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39614/0 (socket says 172.21.15.181:39614) 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb1967fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19f7fe640 1 -- 172.21.15.181:0/4257668104 learned_addr learned my addr 172.21.15.181:0/4257668104 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:51294/0 (socket says 172.21.15.181:51294) 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 -- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 msgr2=0x7fb1a014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 -- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 msgr2=0x7fb1a01562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19effd640 1 -- 172.21.15.181:0/4257668104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a0157120 con 0x7fb1a00d2f00 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.844+0000 7fb19f7fe640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb19effd640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb190033000 tx=0x7fb190004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190037bc0 con 0x7fb1a00d2f00 2024-03-29T04:10:31.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb190046070 con 0x7fb1a00d2f00 2024-03-29T04:10:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190041400 con 0x7fb1a00d2f00 2024-03-29T04:10:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a0157400 con 0x7fb1a00d2f00 2024-03-29T04:10:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.845+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a0154be0 con 0x7fb1a00d2f00 2024-03-29T04:10:31.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.846+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a0075a10 con 0x7fb1a00d2f00 2024-03-29T04:10:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.846+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb190040070 con 0x7fb1a00d2f00 2024-03-29T04:10:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.846+0000 7fb197fff640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 0x7fb18006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.847+0000 7fb1967fc640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 0x7fb18006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.847+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb1900adc00 con 0x7fb1a00d2f00 2024-03-29T04:10:31.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.847+0000 7fb1967fc640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 0x7fb18006e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cb100 tx=0x7fb188009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:31.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:31.851+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb190033030 con 0x7fb1a00d2f00 2024-03-29T04:10:32.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.077+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a0067ed0 con 0x7fb1a00d2f00 2024-03-29T04:10:32.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.078+0000 7fb197fff640 1 -- 172.21.15.181:0/4257668104 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fb19007bab0 con 0x7fb1a00d2f00 2024-03-29T04:10:32.057 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:10:32.057 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:24.646619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:10:32.058 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 16 2024-03-29T04:10:32.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 msgr2=0x7fb18006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 0x7fb18006e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00cb100 tx=0x7fb188009190 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 msgr2=0x7fb1a01546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb190033000 tx=0x7fb190004730 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 shutdown_connections 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1a015e950 0x7fb1a01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb18006c530 0x7fb18006e9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1a00d48c0 0x7fb1a014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 --2- 172.21.15.181:0/4257668104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1a00d2f00 0x7fb1a01546a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 >> 172.21.15.181:0/4257668104 conn(0x7fb1a00676e0 msgr2=0x7fb1a00cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 shutdown_connections 2024-03-29T04:10:32.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.080+0000 7fb1a52c1640 1 -- 172.21.15.181:0/4257668104 wait complete. 2024-03-29T04:10:32.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T04:10:22.645810+0000', 'modified': '2024-03-29T04:10:24.646619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5373': {'gid': 5373, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2447436157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 2447436157}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2447436157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5373, 'qdb_cluster': [5373]} 2024-03-29T04:10:32.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T04:10:32.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T04:10:32.575 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.596+0000 7f70ebdc8640 1 Processor -- start 2024-03-29T04:10:32.576 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.596+0000 7f70ebdc8640 1 -- start start 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ebdc8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70e415e7a0 0x7f70e40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ebdc8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f70e40cec60 0x7f70e40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ebdc8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e40d3120 con 0x7f70e40d3ee0 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ebdc8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f70e40cfc40 con 0x7f70e40cec60 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ebdc8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40d3400 con 0x7f70e415e7a0 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ea365640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f70e40d3ee0 0x7f70e40b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39096/0 (socket says 172.21.15.181:39096) 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70ea365640 1 -- 172.21.15.181:0/1174788616 learned_addr learned my addr 172.21.15.181:0/1174788616 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e9363640 1 --2- 172.21.15.181:0/1174788616 >> v2:172.21.15.187:3300/0 conn(0x7f70e40cec60 0x7f70e40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.580 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e9b64640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70e415e7a0 0x7f70e40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658047151 0 0) 0x7f70e40d3120 con 0x7f70e40d3ee0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70d0003520 con 0x7f70e40d3ee0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1589248853 0 0) 0x7f70d0003520 con 0x7f70e40d3ee0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70e415e7a0 msgr2=0x7f70e40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70e415e7a0 0x7f70e40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 >> v2:172.21.15.187:3300/0 conn(0x7f70e40cec60 msgr2=0x7f70e40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 --2- 172.21.15.181:0/1174788616 >> v2:172.21.15.187:3300/0 conn(0x7f70e40cec60 0x7f70e40d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e40d36e0 con 0x7f70e40d3ee0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f70e00019f0 con 0x7f70e40d3ee0 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 >> v1:172.21.15.181:6789/0 conn(0x7f70e40d3ee0 legacy=0x7f70e40b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d00039b0 0x7f70d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d0004310 0x7f70d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d0004e50 0x7f70d00052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.581 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40cfc40 con 0x7f70d0004310 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d3400 con 0x7f70d0004e50 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.598+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70d000bde0 con 0x7f70d00039b0 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9b64640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d00039b0 0x7f70d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d0004310 0x7f70d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70ea365640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d0004e50 0x7f70d00052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 -- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d00039b0 msgr2=0x7f70d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d00039b0 0x7f70d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 -- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d0004e50 msgr2=0x7f70d00052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d0004e50 0x7f70d00052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9363640 1 -- 172.21.15.181:0/1174788616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d0003520 con 0x7f70d0004310 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70e9b64640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d00039b0 0x7f70d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.599+0000 7f70ea365640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d0004e50 0x7f70d00052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70e9363640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d0004310 0x7f70d0004800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009ac0 tx=0x7f70d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23310b28451e542 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.582 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4039070 con 0x7f70d0004310 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d402c710 con 0x7f70d0004310 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70e8b62640 1 -- 172.21.15.181:0/1174788616 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009690 con 0x7f70d0004310 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 -- 172.21.15.181:0/1174788616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d0004310 msgr2=0x7f70d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/1174788616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d0004310 0x7f70d0004800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009ac0 tx=0x7f70d0005830 comp rx=0 tx=0).stop 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ea365640 1 -- 172.21.15.181:0/1174788616 reap_dead start 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 -- 172.21.15.181:0/1174788616 shutdown_connections 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 -- 172.21.15.181:0/1174788616 >> 172.21.15.181:0/1174788616 conn(0x7f70e40676b0 msgr2=0x7f70e4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 -- 172.21.15.181:0/1174788616 shutdown_connections 2024-03-29T04:10:32.583 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.600+0000 7f70ebdc8640 1 -- 172.21.15.181:0/1174788616 wait complete. 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.601+0000 7f70ebdc8640 1 Processor -- start 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 -- start start 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d0004310 0x7f70e41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 0x7f70e40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d1970 con 0x7f70e40ce1a0 2024-03-29T04:10:32.588 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d17f0 con 0x7f70e40cdd20 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ebdc8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40d1af0 con 0x7f70d0004310 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ea365640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ea365640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39630/0 (socket says 172.21.15.181:39630) 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70ea365640 1 -- 172.21.15.181:0/413447562 learned_addr learned my addr 172.21.15.181:0/413447562 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70e9b64640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d0004310 0x7f70e41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.602+0000 7f70e9363640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 0x7f70e40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 -- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d0004310 msgr2=0x7f70e41527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d0004310 0x7f70e41527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 -- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 msgr2=0x7f70e40ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 0x7f70e40ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 -- 172.21.15.181:0/413447562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e4152d30 con 0x7f70e40ce1a0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70e9363640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 0x7f70e40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ea365640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f70e0001330 tx=0x7f70e0007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70da7fc640 1 -- 172.21.15.181:0/413447562 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc009690 con 0x7f70e40ce1a0 2024-03-29T04:10:32.589 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70da7fc640 1 -- 172.21.15.181:0/413447562 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e0019070 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e41f7ea0 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70da7fc640 1 -- 172.21.15.181:0/413447562 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000a4e0 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.603+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41f8460 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.605+0000 7f70da7fc640 1 -- 172.21.15.181:0/413447562 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f70e0013630 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.605+0000 7f70da7fc640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 0x7f70c806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.605+0000 7f70da7fc640 1 -- 172.21.15.181:0/413447562 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f70e008bce0 con 0x7f70e40ce1a0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.605+0000 7f70e9b64640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 0x7f70c806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.606+0000 7f70e9b64640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 0x7f70c806ea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009b40 tx=0x7f70dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.606+0000 7f70ee9b2740 1 Processor -- start 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.606+0000 7f70ee9b2740 1 -- start start 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9860 0x558cbd69c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd69cd40 0x558cbd6ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558cbd606020 con 0x558cbd6c9c40 2024-03-29T04:10:32.590 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cbd5ae5f0 con 0x558cbd69cd40 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558cbd601fd0 con 0x558cbd6c9860 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39634/0 (socket says 172.21.15.181:39634) 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 -- 172.21.15.181:0/3407682834 learned_addr learned my addr 172.21.15.181:0/3407682834 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9b64640 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9860 0x558cbd69c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70ea365640 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd69cd40 0x558cbd6ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 -- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9860 msgr2=0x558cbd69c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9860 0x558cbd69c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 -- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd69cd40 msgr2=0x558cbd6ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd69cd40 0x558cbd6ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 -- 172.21.15.181:0/3407682834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbd31a2d0 con 0x558cbd6c9c40 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70e9363640 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f70d4037040 tx=0x7f70d402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f90866914ca343 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.607+0000 7f70c77fe640 1 -- 172.21.15.181:0/3407682834 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4045070 con 0x558cbd6c9c40 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70c77fe640 1 -- 172.21.15.181:0/3407682834 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d403a040 con 0x558cbd6c9c40 2024-03-29T04:10:32.591 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70c77fe640 1 -- 172.21.15.181:0/3407682834 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4039070 con 0x558cbd6c9c40 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 -- 172.21.15.181:0/3407682834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 msgr2=0x558cbd6cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f70d4037040 tx=0x7f70d402e000 comp rx=0 tx=0).stop 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 -- 172.21.15.181:0/3407682834 shutdown_connections 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd69cd40 0x558cbd6ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9c40 0x558cbd6cc5f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/3407682834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9860 0x558cbd69c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 -- 172.21.15.181:0/3407682834 >> 172.21.15.181:0/3407682834 conn(0x558cbd69ea70 msgr2=0x558cbd6e75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 -- 172.21.15.181:0/3407682834 shutdown_connections 2024-03-29T04:10:32.592 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.608+0000 7f70ee9b2740 1 -- 172.21.15.181:0/3407682834 wait complete. 2024-03-29T04:10:32.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.609+0000 7f70ee9b2740 1 -- start start 2024-03-29T04:10:32.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.609+0000 7f70ee9b2740 1 Processor -- start 2024-03-29T04:10:32.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558cbd6a1110 con 0x558cbd6c9860 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cbd5c19c0 con 0x558cbd6c9c40 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558cbd6a3950 con 0x558cbd6c5480 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 10 client.0 _ll_register_callbacks cb 0x558cbd5a5fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39648/0 (socket says 172.21.15.181:39648) 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 -- 172.21.15.181:0/397099238 learned_addr learned my addr 172.21.15.181:0/397099238 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9363640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ea365640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 -- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 msgr2=0x558cbd6ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 -- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 msgr2=0x558cbd6c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 -- 172.21.15.181:0/397099238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbd3e46b0 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70e9b64640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0036f0 tx=0x7f70dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcd2ad9263f737 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc029070 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f70dc016880 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d403a040 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.611+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f70dc034050 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 10 client.5574 fetch_fsmap learned FSMap version 16 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x558cbd6cefa0 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d403a040 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f70dc016a80 con 0x558cbd6c9860 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 10 client.5574 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 10 client.5574 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:32.598 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 20 client.5574 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 4 client.5574 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cbd5da4e0 con 0x558cbd6fb680 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70e9363640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70c4ff9640 10 client.5574 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.612+0000 7f70e9363640 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6fba60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917d8dc94ddb7987 server_cookie=1063865faa8808a in_seq=0 out_seq=0 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70c4ff9640 1 -- 172.21.15.181:0/397099238 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x558cbd5da4e0 con 0x558cbd6fb680 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70c4ff9640 10 client.5574 handle_command_reply: tid=0 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 client.5574 shutdown 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 msgr2=0x558cbd6c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0036f0 tx=0x7f70dc004730 comp rx=0 tx=0).stop 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 shutdown_connections 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6fba60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.599 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/397099238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6c2e20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70c4ff9640 0 client.5574 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:32.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 >> 172.21.15.181:0/397099238 conn(0x558cbd69cd40 msgr2=0x558cbd6a07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 shutdown_connections 2024-03-29T04:10:32.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 1 -- 172.21.15.181:0/397099238 wait complete. 2024-03-29T04:10:32.600 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.613+0000 7f70ee9b2740 20 client.5574 trim_cache size 0 max 16384 2024-03-29T04:10:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 Processor -- start 2024-03-29T04:10:32.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 -- start start 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f70e42002e0 con 0x558cbd6c9860 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40b4890 con 0x558cbd6c9c40 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70e41ffde0 con 0x558cbd6c5480 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70e9363640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70e9363640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39658/0 (socket says 172.21.15.181:39658) 2024-03-29T04:10:32.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.632+0000 7f70e9363640 1 -- 172.21.15.181:0/2323069220 learned_addr learned my addr 172.21.15.181:0/2323069220 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 -- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 msgr2=0x558cbd6f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 -- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 msgr2=0x558cbd6e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 -- 172.21.15.181:0/2323069220 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbd698780 con 0x558cbd6c9860 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70e9363640 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70d402c380 tx=0x7f70d4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33809bbb32deaed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70c6ffd640 1 -- 172.21.15.181:0/2323069220 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4045040 con 0x558cbd6c9860 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70c6ffd640 1 -- 172.21.15.181:0/2323069220 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4005990 con 0x558cbd6c9860 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70c6ffd640 1 -- 172.21.15.181:0/2323069220 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4039040 con 0x558cbd6c9860 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2323069220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 msgr2=0x558cbd6e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70d402c380 tx=0x7f70d4008030 comp rx=0 tx=0).stop 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2323069220 shutdown_connections 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9c40 0x558cbd6e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c9860 0x558cbd6e6fd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.633+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2323069220 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c5480 0x558cbd6f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.634+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2323069220 >> 172.21.15.181:0/2323069220 conn(0x558cbd69cd40 msgr2=0x558cbd6c1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.634+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2323069220 shutdown_connections 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.634+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2323069220 wait complete. 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.634+0000 7f70ee9b2740 1 -- start start 2024-03-29T04:10:32.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.634+0000 7f70ee9b2740 1 Processor -- start 2024-03-29T04:10:32.619 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9860 0x558cbd6bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 0x558cbd6be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558cbd6ce650 con 0x558cbd6c5480 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cbd6c2aa0 con 0x558cbd6c9860 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558cbd6cb6a0 con 0x558cbd6c9c40 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ee9b2740 10 client.0 _ll_register_callbacks cb 0x7f70dc014b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39662/0 (socket says 172.21.15.181:39662) 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 -- 172.21.15.181:0/2036359717 learned_addr learned my addr 172.21.15.181:0/2036359717 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9363640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9860 0x558cbd6bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ea365640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 0x558cbd6be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 -- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 msgr2=0x558cbd6be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.620 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 0x558cbd6be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 -- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9860 msgr2=0x558cbd6bdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9860 0x558cbd6bdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70e9b64640 1 -- 172.21.15.181:0/2036359717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbd6c94a0 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.635+0000 7f70ea365640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 0x558cbd6be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70e9b64640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f70dc006c00 tx=0x7f70dc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fa12843fc535ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc029070 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f70dc025690 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d4005990 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f70dc048050 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 10 client.5580 fetch_fsmap learned FSMap version 16 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x558cbd6cb350 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4005990 con 0x558cbd6c5480 2024-03-29T04:10:32.621 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f70dc04b020 con 0x558cbd6c5480 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 10 client.5580 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 10 client.5580 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 20 client.5580 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 4 client.5580 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "status"}) -- 0x558cbd613560 con 0x558cbd6fb680 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.636+0000 7f70e9363640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70c4ff9640 10 client.5580 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70e9363640 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6c8420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ca146142d92227 server_cookie=9dda79046d16b643 in_seq=0 out_seq=0 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70c4ff9640 1 -- 172.21.15.181:0/2036359717 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558cbd613560 con 0x558cbd6fb680 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70c4ff9640 10 client.5580 handle_command_reply: tid=0 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 client.5580 shutdown 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 msgr2=0x558cbd762eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f70dc006c00 tx=0x7f70dc009190 comp rx=0 tx=0).stop 2024-03-29T04:10:32.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 shutdown_connections 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558cbd6fb680 0x558cbd6c8420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558cbd6c9c40 0x558cbd6be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70c4ff9640 0 client.5580 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cbd6c9860 0x558cbd6bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 --2- 172.21.15.181:0/2036359717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cbd6c5480 0x558cbd762eb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 >> 172.21.15.181:0/2036359717 conn(0x558cbd69cd40 msgr2=0x558cbd70ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 shutdown_connections 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 1 -- 172.21.15.181:0/2036359717 wait complete. 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.637+0000 7f70ee9b2740 20 client.5580 trim_cache size 0 max 16384 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stdout: "cluster_fsid": "fab9a006-093b-45b9-834f-2720bd7aa224", 2024-03-29T04:10:32.623 INFO:teuthology.orchestra.run.smithi181.stdout: "whoami": 0, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 5373, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "want_state": "up:active", 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "up:active", 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs", 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "rank_uptime": 9.3487661919999994, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "mdsmap_epoch": 16, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch": 39, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_epoch_barrier": 39, 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout: "uptime": 16.446813175999999 2024-03-29T04:10:32.624 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 msgr2=0x7f70c806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 0x7f70c806ea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009b40 tx=0x7f70dc008040 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 msgr2=0x7f70e40ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f70e0001330 tx=0x7f70e0007060 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 shutdown_connections 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70e40ce1a0 0x7f70e40ca580 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e40cdd20 0x7f70e40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f70c806c560 0x7f70c806ea20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 --2- 172.21.15.181:0/413447562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70d0004310 0x7f70e41527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 >> 172.21.15.181:0/413447562 conn(0x7f70e40676b0 msgr2=0x7f70e40b9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 shutdown_connections 2024-03-29T04:10:32.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.638+0000 7f70ebdc8640 1 -- 172.21.15.181:0/413447562 wait complete. 2024-03-29T04:10:32.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:32.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.754+0000 7f8035c7b640 1 Processor -- start 2024-03-29T04:10:32.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.754+0000 7f8035c7b640 1 -- start start 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f8035c7b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f8035c7b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 0x7f80300d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f8035c7b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80300d36f0 con 0x7f80300d2f00 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f8035c7b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f80300d3cb0 con 0x7f80300d48c0 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f8035c7b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80300d4d00 con 0x7f803015e950 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f802f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f802effd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 0x7f80300d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f802f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:54782/0 (socket says 172.21.15.181:54782) 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.755+0000 7f802f7fe640 1 -- 172.21.15.181:0/3789331819 learned_addr learned my addr 172.21.15.181:0/3789331819 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802f7fe640 1 -- 172.21.15.181:0/3789331819 >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 msgr2=0x7f80300d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:32.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3789331819 >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 0x7f80300d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802f7fe640 1 -- 172.21.15.181:0/3789331819 >> v1:172.21.15.187:6789/0 conn(0x7f80300d2f00 legacy=0x7f80300cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802f7fe640 1 -- 172.21.15.181:0/3789331819 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80300d39d0 con 0x7f803015e950 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802effd640 1 --2- 172.21.15.181:0/3789331819 >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 0x7f80300d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802e7fc640 1 -- 172.21.15.181:0/3789331819 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129625857 0 0) 0x7f80300d36f0 con 0x7f80300d2f00 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3789331819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8020009e50 tx=0x7f802002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81dae4918c3ed74d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802e7fc640 1 -- 172.21.15.181:0/3789331819 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802002cb90 con 0x7f803015e950 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.756+0000 7f802e7fc640 1 -- 172.21.15.181:0/3789331819 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802002cd30 con 0x7f803015e950 2024-03-29T04:10:32.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f802e7fc640 1 -- 172.21.15.181:0/3789331819 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020034610 con 0x7f803015e950 2024-03-29T04:10:32.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 -- 172.21.15.181:0/3789331819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 msgr2=0x7f80300d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.738 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3789331819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8020009e50 tx=0x7f802002bff0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 -- 172.21.15.181:0/3789331819 shutdown_connections 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3789331819 >> v2:172.21.15.187:3300/0 conn(0x7f80300d48c0 0x7f80300d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3789331819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f803015e950 0x7f80300d4380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 -- 172.21.15.181:0/3789331819 >> 172.21.15.181:0/3789331819 conn(0x7f80300676e0 msgr2=0x7f8030067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 -- 172.21.15.181:0/3789331819 shutdown_connections 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.757+0000 7f8035c7b640 1 -- 172.21.15.181:0/3789331819 wait complete. 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.759+0000 7f8035c7b640 1 Processor -- start 2024-03-29T04:10:32.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.759+0000 7f8035c7b640 1 -- start start 2024-03-29T04:10:32.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 0x7f80300ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d48c0 0x7f803015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80300d3390 con 0x7f803015e950 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80300b7da0 con 0x7f80300d2f00 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f8035c7b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f803015ed30 con 0x7f80300d48c0 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f802ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f802ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39664/0 (socket says 172.21.15.181:39664) 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.760+0000 7f802ffff640 1 -- 172.21.15.181:0/3576218484 learned_addr learned my addr 172.21.15.181:0/3576218484 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 0x7f80300ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802effd640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d48c0 0x7f803015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 -- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d48c0 msgr2=0x7f803015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d48c0 0x7f803015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 -- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 msgr2=0x7f80300ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 0x7f80300ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 -- 172.21.15.181:0/3576218484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803015ffd0 con 0x7f803015e950 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 0x7f80300ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f802ffff640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f802800c3e0 tx=0x7f802800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f803015e950 2024-03-29T04:10:32.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80280035f0 con 0x7f803015e950 2024-03-29T04:10:32.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80301602a0 con 0x7f803015e950 2024-03-29T04:10:32.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280039e0 con 0x7f803015e950 2024-03-29T04:10:32.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.761+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8030160860 con 0x7f803015e950 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.763+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8030160ba0 con 0x7f803015e950 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.763+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f802801a070 con 0x7f803015e950 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.763+0000 7f8027fff640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 0x7f801006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.763+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 0x7f801006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.763+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f802808ab80 con 0x7f803015e950 2024-03-29T04:10:32.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.764+0000 7f802f7fe640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 0x7f801006e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8020035000 tx=0x7f8020030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:32.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:32.768+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8030160ba0 con 0x7f803015e950 2024-03-29T04:10:33.011 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.032+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8030067c60 con 0x7f803015e950 2024-03-29T04:10:33.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.033+0000 7f8027fff640 1 -- 172.21.15.181:0/3576218484 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f8028058930 con 0x7f803015e950 2024-03-29T04:10:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 msgr2=0x7f801006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 0x7f801006e9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8020035000 tx=0x7f8020030000 comp rx=0 tx=0).stop 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 msgr2=0x7f803014c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f802800c3e0 tx=0x7f802800c8a0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 shutdown_connections 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f803015e950 0x7f803014c690 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f801006c530 0x7f801006e9f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80300d48c0 0x7f803015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 --2- 172.21.15.181:0/3576218484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80300d2f00 0x7f80300ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.035+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 >> 172.21.15.181:0/3576218484 conn(0x7f80300676e0 msgr2=0x7f80300b89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.036+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 shutdown_connections 2024-03-29T04:10:33.016 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.036+0000 7f8035c7b640 1 -- 172.21.15.181:0/3576218484 wait complete. 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:33.029 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:10:33.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-29T04:10:33.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:33.030 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:10:33.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.030 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:33.045 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 6e:d7:dc:da:09:7c brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::9851:47ff:fed2:ccc4/64 scope link 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 6e:d7:dc:da:09:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::6cd7:dcff:feda:97c/64 scope link 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether b2:63:4b:17:b5:30 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::b063:4bff:fe17:b530/64 scope link 2024-03-29T04:10:33.047 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:33.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.048 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:33.110 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-03-29T04:10:33.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.112 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-03-29T04:10:33.168 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-03-29T04:10:33.203 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:33.203 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:33.203 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:33.203 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:33.203 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:33.203 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-03-29T04:10:33.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:33.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:33.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:33.204 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-03-29T04:10:33.256 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:33.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:33.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.304 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:10:33.318 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:10:33.318 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:10:33.318 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:10:33.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-03-29T04:10:33.387 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:33.387 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:33.442 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T04:10:33.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:33.442 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:33.497 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-29T04:10:33.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:33.582 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:33.583 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:10:33.590 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:33.590 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:10:33.590 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:10:33.590 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 0@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:10:33.590 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:10:33.591 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-29T04:10:33.591 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:10:33.591 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:10:33.591 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:10:33.591 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:10:33.593 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:10:33.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:33.593 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T04:10:33.620 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:10:33.729 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:10:33.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:33.741 INFO:teuthology.orchestra.run.smithi181.stdout:client5586 2024-03-29T04:10:33.741 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:10:33.741 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:33.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T04:10:33.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.920+0000 7f4472eb2640 1 Processor -- start 2024-03-29T04:10:33.900 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.920+0000 7f4472eb2640 1 -- start start 2024-03-29T04:10:33.902 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4472eb2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f446c0d4a90 0x7f446c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4472eb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f446c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4472eb2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446c0d36f0 con 0x7f446c15e950 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4472eb2640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f446c0d3cb0 con 0x7f446c0d4a90 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4472eb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f446c0d39d0 con 0x7f446c0d2f00 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4470c4e640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f446c15e950 0x7f446c0d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:39106/0 (socket says 172.21.15.181:39106) 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.921+0000 7f4470c4e640 1 -- 172.21.15.181:0/1675433164 learned_addr learned my addr 172.21.15.181:0/1675433164 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626519440 0 0) 0x7f446c0d36f0 con 0x7f446c15e950 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446bfff640 1 --2- 172.21.15.181:0/1675433164 >> v2:172.21.15.187:3300/0 conn(0x7f446c0d4a90 0x7f446c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f447144f640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f446c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f445c003520 con 0x7f446c15e950 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155323868 0 0) 0x7f445c003520 con 0x7f446c15e950 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 msgr2=0x7f446c0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f446c0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.903 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 >> v2:172.21.15.187:3300/0 conn(0x7f446c0d4a90 msgr2=0x7f446c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 --2- 172.21.15.181:0/1675433164 >> v2:172.21.15.187:3300/0 conn(0x7f446c0d4a90 0x7f446c0d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c0b7ac0 con 0x7f446c15e950 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f4460001670 con 0x7f446c15e950 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 >> v1:172.21.15.181:6789/0 conn(0x7f446c15e950 legacy=0x7f446c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445c0039b0 0x7f445c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f445c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c004d40 0x7f445c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d3cb0 con 0x7f445c004d40 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d39d0 con 0x7f445c0039b0 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f445c00b9c0 con 0x7f446c0d2f00 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f4470c4e640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445c0039b0 0x7f445c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f447144f640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c004d40 0x7f445c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.922+0000 7f446bfff640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f445c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 -- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 msgr2=0x7f445c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f446c0d2f00 0x7f445c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 -- 172.21.15.181:0/1675433164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c004d40 msgr2=0x7f445c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c004d40 0x7f445c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 -- 172.21.15.181:0/1675433164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c003520 con 0x7f445c0039b0 2024-03-29T04:10:33.905 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.923+0000 7f4470c4e640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445c0039b0 0x7f445c003dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f446000c210 tx=0x7f445c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d9d0a326b9f3750 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460007d50 con 0x7f445c0039b0 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4460030070 con 0x7f445c0039b0 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f446b7fe640 1 -- 172.21.15.181:0/1675433164 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446002cd80 con 0x7f445c0039b0 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 -- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445c0039b0 msgr2=0x7f445c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 --2- 172.21.15.181:0/1675433164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f445c0039b0 0x7f445c003dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f446000c210 tx=0x7f445c005830 comp rx=0 tx=0).stop 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f447144f640 1 -- 172.21.15.181:0/1675433164 reap_dead start 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 -- 172.21.15.181:0/1675433164 shutdown_connections 2024-03-29T04:10:33.908 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 -- 172.21.15.181:0/1675433164 >> 172.21.15.181:0/1675433164 conn(0x7f446c0676e0 msgr2=0x7f446c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 -- 172.21.15.181:0/1675433164 shutdown_connections 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.924+0000 7f4472eb2640 1 -- 172.21.15.181:0/1675433164 wait complete. 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 Processor -- start 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 -- start start 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 0x7f446c14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 0x7f446c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f446c15ed30 con 0x7f446c0cd490 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d33b0 con 0x7f446c14f050 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.926+0000 7f4472eb2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f446c15eeb0 con 0x7f445c0039b0 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39682/0 (socket says 172.21.15.181:39682) 2024-03-29T04:10:33.909 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 -- 172.21.15.181:0/406418374 learned_addr learned my addr 172.21.15.181:0/406418374 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f4470c4e640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 0x7f446c14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f446bfff640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 0x7f446c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 -- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 msgr2=0x7f446c14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 0x7f446c14c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 -- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 msgr2=0x7f446c14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 0x7f446c14f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 -- 172.21.15.181:0/406418374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c14cd00 con 0x7f446c0cd490 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f446bfff640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 0x7f446c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f4470c4e640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 0x7f446c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f447144f640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f446c162510 tx=0x7f446400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446400fe10 con 0x7f446c0cd490 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446401d070 con 0x7f446c0cd490 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c14d180 con 0x7f446c0cd490 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464015670 con 0x7f446c0cd490 2024-03-29T04:10:33.910 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.927+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c14fdc0 con 0x7f446c0cd490 2024-03-29T04:10:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.928+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446c0cde50 con 0x7f446c0cd490 2024-03-29T04:10:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.929+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4464015810 con 0x7f446c0cd490 2024-03-29T04:10:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.929+0000 7f44697fa640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 0x7f445006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.929+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f446408c1b0 con 0x7f446c0cd490 2024-03-29T04:10:33.911 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.929+0000 7f4470c4e640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 0x7f445006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:33.912 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.930+0000 7f4470c4e640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 0x7f445006ea20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4460034000 tx=0x7f4460007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:33.913 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:33.934+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4464010040 con 0x7f446c0cd490 2024-03-29T04:10:34.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.184+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f446c075a10 con 0x7f446c0cd490 2024-03-29T04:10:34.163 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.184+0000 7f44697fa640 1 -- 172.21.15.181:0/406418374 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f446405a640 con 0x7f446c0cd490 2024-03-29T04:10:34.164 INFO:teuthology.orchestra.run.smithi181.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 msgr2=0x7f445006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 0x7f445006ea20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4460034000 tx=0x7f4460007000 comp rx=0 tx=0).stop 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 msgr2=0x7f446c0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f446c162510 tx=0x7f446400f580 comp rx=0 tx=0).stop 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 shutdown_connections 2024-03-29T04:10:34.166 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f446c0cd490 0x7f446c0cd910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f446c14f050 0x7f446c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f445006c560 0x7f445006ea20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 --2- 172.21.15.181:0/406418374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f445c0039b0 0x7f446c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 >> 172.21.15.181:0/406418374 conn(0x7f446c0676e0 msgr2=0x7f446c0ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 shutdown_connections 2024-03-29T04:10:34.167 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:34.186+0000 7f4472eb2640 1 -- 172.21.15.181:0/406418374 wait complete. 2024-03-29T04:10:34.176 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T04:10:34.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T04:10:34.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:10:34.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.177 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-03-29T04:10:34.196 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T04:10:34.196 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T04:10:34.196 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T04:10:34.196 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:34.197 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42661sec preferred_lft 42661sec 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 4a:c0:2f:62:53:2f brd ff:ff:ff:ff:ff:ff 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::78a9:f5ff:fe74:e7b6/64 scope link 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether e2:ad:e4:80:6e:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::e0ad:e4ff:fe80:6ec6/64 scope link 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.198 INFO:teuthology.orchestra.run.smithi187.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T04:10:34.199 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 4a:c0:2f:62:53:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:10:34.199 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::48c0:2fff:fe62:532f/64 scope link 2024-03-29T04:10:34.199 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-03-29T04:10:34.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.199 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T04:10:34.267 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-03-29T04:10:34.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.269 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-03-29T04:10:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-03-29T04:10:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T04:10:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T04:10:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T04:10:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T04:10:34.328 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-29T04:10:34.329 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-03-29T04:10:34.329 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T04:10:34.329 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T04:10:34.329 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T04:10:34.329 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-03-29T04:10:34.420 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:10:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-29T04:10:34.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:34.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.468 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:10:34.483 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:10:34.483 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:10:34.484 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:10:34.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.484 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-03-29T04:10:34.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:34.552 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:34.606 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T04:10:34.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T04:10:34.606 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:34.660 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-29T04:10:34.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:34.744 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:34.744 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.181:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with new device syntax: 1@fab9a006-093b-45b9-834f-2720bd7aa224.cephfs=/ 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.181:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-03-29T04:10:34.752 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-03-29T04:10:34.753 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-29T04:10:34.754 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-29T04:10:34.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:34.754 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T04:10:34.782 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-29T04:10:34.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:10:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-29T04:10:34.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-03-29T04:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-29T04:10:34.877 INFO:teuthology.orchestra.run.smithi187.stdout:client5491 2024-03-29T04:10:34.877 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:10:34.877 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:10:34.893 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-29T04:10:34.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-29T04:10:34.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.020+0000 7feec1ffb640 1 Processor -- start 2024-03-29T04:10:34.999 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.020+0000 7feec1ffb640 1 -- start start 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feebc15e440 0x7feebc0b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc0d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feebc0d3ef0 0x7feebc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7feebc0d3520 con 0x7feebc15e440 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feebc0d2910 con 0x7feebc0d3ef0 2024-03-29T04:10:35.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feec1ffb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feebc0d2bf0 con 0x7feebc06d4a0 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feebc15e440 0x7feebc0b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feebc15e440 0x7feebc0b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39696/0 (socket says 172.21.15.181:39696) 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 -- 172.21.15.181:0/3785534416 learned_addr learned my addr 172.21.15.181:0/3785534416 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebaffd640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc0d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebbfff640 1 --2- 172.21.15.181:0/3785534416 >> v2:172.21.15.187:3300/0 conn(0x7feebc0d3ef0 0x7feebc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 -- 172.21.15.181:0/3785534416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 msgr2=0x7feebc0d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc0d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 -- 172.21.15.181:0/3785534416 >> v2:172.21.15.187:3300/0 conn(0x7feebc0d3ef0 msgr2=0x7feebc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 --2- 172.21.15.181:0/3785534416 >> v2:172.21.15.187:3300/0 conn(0x7feebc0d3ef0 0x7feebc0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.021+0000 7feebb7fe640 1 -- 172.21.15.181:0/3785534416 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0d2ed0 con 0x7feebc15e440 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.022+0000 7feebb7fe640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc15e440 0x7feebc0b9ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feebc0d0280 tx=0x7feeac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfa3f4224b8c2db server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.022+0000 7feeba7fc640 1 -- 172.21.15.181:0/3785534416 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac034070 con 0x7feebc15e440 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.022+0000 7feeba7fc640 1 -- 172.21.15.181:0/3785534416 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac030860 con 0x7feebc15e440 2024-03-29T04:10:35.002 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.022+0000 7feeba7fc640 1 -- 172.21.15.181:0/3785534416 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac030c30 con 0x7feebc15e440 2024-03-29T04:10:35.003 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 -- 172.21.15.181:0/3785534416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc15e440 msgr2=0x7feebc0b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc15e440 0x7feebc0b9ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feebc0d0280 tx=0x7feeac02b9a0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 -- 172.21.15.181:0/3785534416 shutdown_connections 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 --2- 172.21.15.181:0/3785534416 >> v2:172.21.15.187:3300/0 conn(0x7feebc0d3ef0 0x7feebc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc0d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 --2- 172.21.15.181:0/3785534416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc15e440 0x7feebc0b9ce0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 -- 172.21.15.181:0/3785534416 >> 172.21.15.181:0/3785534416 conn(0x7feebc0676b0 msgr2=0x7feebc067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 -- 172.21.15.181:0/3785534416 shutdown_connections 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.023+0000 7feec1ffb640 1 -- 172.21.15.181:0/3785534416 wait complete. 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.024+0000 7feec1ffb640 1 Processor -- start 2024-03-29T04:10:35.004 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.024+0000 7feec1ffb640 1 -- start start 2024-03-29T04:10:35.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feebc15e440 0x7feebc154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.006 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feebc15e820 con 0x7feebc0c7e40 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0ba220 con 0x7feebc15e440 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feebc15e9a0 con 0x7feebc06d4a0 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39710/0 (socket says 172.21.15.181:39710) 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 -- 172.21.15.181:0/31789373 learned_addr learned my addr 172.21.15.181:0/31789373 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebb7fe640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebaffd640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feebc15e440 0x7feebc154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 -- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 msgr2=0x7feebc1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc1f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 -- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feebc15e440 msgr2=0x7feebc154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feebc15e440 0x7feebc154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 -- 172.21.15.181:0/31789373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc147150 con 0x7feebc0c7e40 2024-03-29T04:10:35.007 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feebbfff640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feeb400ef90 tx=0x7feeb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4018020 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4014480 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.025+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc0ca670 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.026+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4014870 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.026+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc0cabf0 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.027+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc075900 con 0x7feebc0c7e40 2024-03-29T04:10:35.008 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.027+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feeb4005d10 con 0x7feebc0c7e40 2024-03-29T04:10:35.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.028+0000 7feea3fff640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 0x7feea806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:35.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.028+0000 7feebb7fe640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 0x7feea806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:35.009 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.028+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7feeb408c440 con 0x7feebc0c7e40 2024-03-29T04:10:35.010 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.028+0000 7feebb7fe640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 0x7feea806e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feeac036000 tx=0x7feeac03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:35.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.033+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feeb4022030 con 0x7feebc0c7e40 2024-03-29T04:10:35.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.299+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7feebc15a100 con 0x7feebc0c7e40 2024-03-29T04:10:35.310 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.331+0000 7feea3fff640 1 -- 172.21.15.181:0/31789373 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v17) ==== 165+0+0 (secure 0 0 0) 0x7feeb4010040 con 0x7feebc0c7e40 2024-03-29T04:10:35.311 INFO:teuthology.orchestra.run.smithi181.stderr:inline data enabled 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 msgr2=0x7feea806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 0x7feea806e9f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feeac036000 tx=0x7feeac03b040 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 msgr2=0x7feebc0c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feeb400ef90 tx=0x7feeb400c490 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 shutdown_connections 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feebc0c7e40 0x7feebc0c99a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feebc15e440 0x7feebc154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7feea806c530 0x7feea806e9f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 --2- 172.21.15.181:0/31789373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feebc06d4a0 0x7feebc1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:35.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.333+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 >> 172.21.15.181:0/31789373 conn(0x7feebc0676b0 msgr2=0x7feebc0b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.334+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 shutdown_connections 2024-03-29T04:10:35.314 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:35.334+0000 7feec1ffb640 1 -- 172.21.15.181:0/31789373 wait complete. 2024-03-29T04:10:35.324 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-29T04:10:35.324 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:10:35.510 INFO:teuthology.orchestra.run.smithi181.stderr:--2024-03-29 04:10:35-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-29T04:10:35.510 INFO:teuthology.orchestra.run.smithi181.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-29T04:10:35.546 INFO:teuthology.orchestra.run.smithi181.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-29T04:10:35.582 INFO:teuthology.orchestra.run.smithi181.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-29T04:10:35.582 INFO:teuthology.orchestra.run.smithi181.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-29T04:10:35.585 INFO:teuthology.orchestra.run.smithi181.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-03-29T04:10:35.587 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:10:35.678 INFO:teuthology.orchestra.run.smithi181.stderr: 0K .......... .......... .......... .......... .......... 0% 559K 5m2s 2024-03-29T04:10:35.713 INFO:teuthology.orchestra.run.smithi181.stderr: 50K .......... .......... .......... .......... .......... 0% 1.38M 3m30s 2024-03-29T04:10:35.713 INFO:teuthology.orchestra.run.smithi181.stderr: 100K .......... .......... .......... .......... .......... 0% 48.6M 2m21s 2024-03-29T04:10:35.750 INFO:teuthology.orchestra.run.smithi181.stderr: 150K .......... .......... .......... .......... .......... 0% 1.30M 2m17s 2024-03-29T04:10:35.786 INFO:teuthology.orchestra.run.smithi181.stderr: 200K .......... .......... .......... .......... .......... 0% 1.40M 2m13s 2024-03-29T04:10:35.786 INFO:teuthology.orchestra.run.smithi181.stderr: 250K .......... .......... .......... .......... .......... 0% 25.5M 1m52s 2024-03-29T04:10:35.822 INFO:teuthology.orchestra.run.smithi181.stderr: 300K .......... .......... .......... .......... .......... 0% 1.44M 1m52s 2024-03-29T04:10:35.822 INFO:teuthology.orchestra.run.smithi181.stderr: 350K .......... .......... .......... .......... .......... 0% 21.7M 99s 2024-03-29T04:10:35.857 INFO:teuthology.orchestra.run.smithi181.stderr: 400K .......... .......... .......... .......... .......... 0% 1.47M 1m41s 2024-03-29T04:10:35.895 INFO:teuthology.orchestra.run.smithi181.stderr: 450K .......... .......... .......... .......... .......... 0% 1.37M 1m43s 2024-03-29T04:10:35.895 INFO:teuthology.orchestra.run.smithi181.stderr: 500K .......... .......... .......... .......... .......... 0% 93.5M 93s 2024-03-29T04:10:35.931 INFO:teuthology.orchestra.run.smithi181.stderr: 550K .......... .......... .......... .......... .......... 0% 1.39M 95s 2024-03-29T04:10:35.931 INFO:teuthology.orchestra.run.smithi181.stderr: 600K .......... .......... .......... .......... .......... 0% 115M 88s 2024-03-29T04:10:35.966 INFO:teuthology.orchestra.run.smithi181.stderr: 650K .......... .......... .......... .......... .......... 0% 1.39M 90s 2024-03-29T04:10:35.966 INFO:teuthology.orchestra.run.smithi181.stderr: 700K .......... .......... .......... .......... .......... 0% 112M 84s 2024-03-29T04:10:36.002 INFO:teuthology.orchestra.run.smithi181.stderr: 750K .......... .......... .......... .......... .......... 0% 1.39M 86s 2024-03-29T04:10:36.002 INFO:teuthology.orchestra.run.smithi181.stderr: 800K .......... .......... .......... .......... .......... 0% 112M 81s 2024-03-29T04:10:36.038 INFO:teuthology.orchestra.run.smithi181.stderr: 850K .......... .......... .......... .......... .......... 0% 1.39M 83s 2024-03-29T04:10:36.038 INFO:teuthology.orchestra.run.smithi181.stderr: 900K .......... .......... .......... .......... .......... 0% 112M 79s 2024-03-29T04:10:36.038 INFO:teuthology.orchestra.run.smithi181.stderr: 950K .......... .......... .......... .......... .......... 0% 43.8M 75s 2024-03-29T04:10:36.074 INFO:teuthology.orchestra.run.smithi181.stderr: 1000K .......... .......... .......... .......... .......... 0% 1.43M 77s 2024-03-29T04:10:36.075 INFO:teuthology.orchestra.run.smithi181.stderr: 1050K .......... .......... .......... .......... .......... 0% 67.4M 74s 2024-03-29T04:10:36.110 INFO:teuthology.orchestra.run.smithi181.stderr: 1100K .......... .......... .......... .......... .......... 0% 1.41M 75s 2024-03-29T04:10:36.110 INFO:teuthology.orchestra.run.smithi181.stderr: 1150K .......... .......... .......... .......... .......... 0% 118M 72s 2024-03-29T04:10:36.144 INFO:teuthology.orchestra.run.smithi181.stderr: 1200K .......... .......... .......... .......... .......... 0% 57.7M 70s 2024-03-29T04:10:36.144 INFO:teuthology.orchestra.run.smithi181.stderr: 1250K .......... .......... .......... .......... .......... 0% 1.42M 71s 2024-03-29T04:10:36.144 INFO:teuthology.orchestra.run.smithi181.stderr: 1300K .......... .......... .......... .......... .......... 0% 102M 69s 2024-03-29T04:10:36.181 INFO:teuthology.orchestra.run.smithi181.stderr: 1350K .......... .......... .......... .......... .......... 0% 1.40M 70s 2024-03-29T04:10:36.181 INFO:teuthology.orchestra.run.smithi181.stderr: 1400K .......... .......... .......... .......... .......... 0% 113M 68s 2024-03-29T04:10:36.181 INFO:teuthology.orchestra.run.smithi181.stderr: 1450K .......... .......... .......... .......... .......... 0% 108M 66s 2024-03-29T04:10:36.216 INFO:teuthology.orchestra.run.smithi181.stderr: 1500K .......... .......... .......... .......... .......... 0% 1.41M 67s 2024-03-29T04:10:36.217 INFO:teuthology.orchestra.run.smithi181.stderr: 1550K .......... .......... .......... .......... .......... 0% 99.9M 65s 2024-03-29T04:10:36.217 INFO:teuthology.orchestra.run.smithi181.stderr: 1600K .......... .......... .......... .......... .......... 0% 107M 63s 2024-03-29T04:10:36.252 INFO:teuthology.orchestra.run.smithi181.stderr: 1650K .......... .......... .......... .......... .......... 1% 1.41M 65s 2024-03-29T04:10:36.252 INFO:teuthology.orchestra.run.smithi181.stderr: 1700K .......... .......... .......... .......... .......... 1% 110M 63s 2024-03-29T04:10:36.252 INFO:teuthology.orchestra.run.smithi181.stderr: 1750K .......... .......... .......... .......... .......... 1% 106M 61s 2024-03-29T04:10:36.288 INFO:teuthology.orchestra.run.smithi181.stderr: 1800K .......... .......... .......... .......... .......... 1% 1.41M 63s 2024-03-29T04:10:36.288 INFO:teuthology.orchestra.run.smithi181.stderr: 1850K .......... .......... .......... .......... .......... 1% 92.3M 61s 2024-03-29T04:10:36.288 INFO:teuthology.orchestra.run.smithi181.stderr: 1900K .......... .......... .......... .......... .......... 1% 136M 60s 2024-03-29T04:10:36.322 INFO:teuthology.orchestra.run.smithi181.stderr: 1950K .......... .......... .......... .......... .......... 1% 1.33M 61s 2024-03-29T04:10:36.322 INFO:teuthology.orchestra.run.smithi181.stderr: 2000K .......... .......... .......... .......... .......... 1% 46.1M 60s 2024-03-29T04:10:36.322 INFO:teuthology.orchestra.run.smithi181.stderr: 2050K .......... .......... .......... .......... .......... 1% 396M 58s 2024-03-29T04:10:36.362 INFO:teuthology.orchestra.run.smithi181.stderr: 2100K .......... .......... .......... .......... .......... 1% 1.37M 60s 2024-03-29T04:10:36.362 INFO:teuthology.orchestra.run.smithi181.stderr: 2150K .......... .......... .......... .......... .......... 1% 120M 58s 2024-03-29T04:10:36.362 INFO:teuthology.orchestra.run.smithi181.stderr: 2200K .......... .......... .......... .......... .......... 1% 107M 57s 2024-03-29T04:10:36.362 INFO:teuthology.orchestra.run.smithi181.stderr: 2250K .......... .......... .......... .......... .......... 1% 109M 56s 2024-03-29T04:10:36.401 INFO:teuthology.orchestra.run.smithi181.stderr: 2300K .......... .......... .......... .......... .......... 1% 1.42M 57s 2024-03-29T04:10:36.401 INFO:teuthology.orchestra.run.smithi181.stderr: 2350K .......... .......... .......... .......... .......... 1% 110M 56s 2024-03-29T04:10:36.401 INFO:teuthology.orchestra.run.smithi181.stderr: 2400K .......... .......... .......... .......... .......... 1% 110M 55s 2024-03-29T04:10:36.425 INFO:teuthology.orchestra.run.smithi181.stderr: 2450K .......... .......... .......... .......... .......... 1% 1.48M 56s 2024-03-29T04:10:36.436 INFO:teuthology.orchestra.run.smithi181.stderr: 2500K .......... .......... .......... .......... .......... 1% 28.1M 55s 2024-03-29T04:10:36.436 INFO:teuthology.orchestra.run.smithi181.stderr: 2550K .......... .......... .......... .......... .......... 1% 111M 54s 2024-03-29T04:10:36.436 INFO:teuthology.orchestra.run.smithi181.stderr: 2600K .......... .......... .......... .......... .......... 1% 110M 53s 2024-03-29T04:10:36.471 INFO:teuthology.orchestra.run.smithi181.stderr: 2650K .......... .......... .......... .......... .......... 1% 1.43M 54s 2024-03-29T04:10:36.472 INFO:teuthology.orchestra.run.smithi181.stderr: 2700K .......... .......... .......... .......... .......... 1% 112M 53s 2024-03-29T04:10:36.472 INFO:teuthology.orchestra.run.smithi181.stderr: 2750K .......... .......... .......... .......... .......... 1% 117M 52s 2024-03-29T04:10:36.472 INFO:teuthology.orchestra.run.smithi181.stderr: 2800K .......... .......... .......... .......... .......... 1% 109M 51s 2024-03-29T04:10:36.508 INFO:teuthology.orchestra.run.smithi181.stderr: 2850K .......... .......... .......... .......... .......... 1% 1.48M 52s 2024-03-29T04:10:36.508 INFO:teuthology.orchestra.run.smithi181.stderr: 2900K .......... .......... .......... .......... .......... 1% 28.5M 51s 2024-03-29T04:10:36.508 INFO:teuthology.orchestra.run.smithi181.stderr: 2950K .......... .......... .......... .......... .......... 1% 109M 50s 2024-03-29T04:10:36.509 INFO:teuthology.orchestra.run.smithi181.stderr: 3000K .......... .......... .......... .......... .......... 1% 114M 50s 2024-03-29T04:10:36.509 INFO:teuthology.orchestra.run.smithi181.stderr: 3050K .......... .......... .......... .......... .......... 1% 111M 49s 2024-03-29T04:10:36.542 INFO:teuthology.orchestra.run.smithi181.stderr: 3100K .......... .......... .......... .......... .......... 1% 1.43M 50s 2024-03-29T04:10:36.542 INFO:teuthology.orchestra.run.smithi181.stderr: 3150K .......... .......... .......... .......... .......... 1% 110M 49s 2024-03-29T04:10:36.542 INFO:teuthology.orchestra.run.smithi181.stderr: 3200K .......... .......... .......... .......... .......... 1% 114M 48s 2024-03-29T04:10:36.543 INFO:teuthology.orchestra.run.smithi181.stderr: 3250K .......... .......... .......... .......... .......... 1% 111M 48s 2024-03-29T04:10:36.578 INFO:teuthology.orchestra.run.smithi181.stderr: 3300K .......... .......... .......... .......... .......... 1% 1.44M 49s 2024-03-29T04:10:36.579 INFO:teuthology.orchestra.run.smithi181.stderr: 3350K .......... .......... .......... .......... .......... 2% 112M 48s 2024-03-29T04:10:36.579 INFO:teuthology.orchestra.run.smithi181.stderr: 3400K .......... .......... .......... .......... .......... 2% 111M 47s 2024-03-29T04:10:36.579 INFO:teuthology.orchestra.run.smithi181.stderr: 3450K .......... .......... .......... .......... .......... 2% 96.2M 46s 2024-03-29T04:10:36.579 INFO:teuthology.orchestra.run.smithi181.stderr: 3500K .......... .......... .......... .......... .......... 2% 105M 46s 2024-03-29T04:10:36.615 INFO:teuthology.orchestra.run.smithi181.stderr: 3550K .......... .......... .......... .......... .......... 2% 1.44M 47s 2024-03-29T04:10:36.615 INFO:teuthology.orchestra.run.smithi181.stderr: 3600K .......... .......... .......... .......... .......... 2% 74.3M 46s 2024-03-29T04:10:36.615 INFO:teuthology.orchestra.run.smithi181.stderr: 3650K .......... .......... .......... .......... .......... 2% 89.4M 45s 2024-03-29T04:10:36.615 INFO:teuthology.orchestra.run.smithi181.stderr: 3700K .......... .......... .......... .......... .......... 2% 110M 45s 2024-03-29T04:10:36.615 INFO:teuthology.orchestra.run.smithi181.stderr: 3750K .......... .......... .......... .......... .......... 2% 112M 44s 2024-03-29T04:10:36.653 INFO:teuthology.orchestra.run.smithi181.stderr: 3800K .......... .......... .......... .......... .......... 2% 1.46M 45s 2024-03-29T04:10:36.653 INFO:teuthology.orchestra.run.smithi181.stderr: 3850K .......... .......... .......... .......... .......... 2% 74.6M 45s 2024-03-29T04:10:36.653 INFO:teuthology.orchestra.run.smithi181.stderr: 3900K .......... .......... .......... .......... .......... 2% 84.4M 44s 2024-03-29T04:10:36.653 INFO:teuthology.orchestra.run.smithi181.stderr: 3950K .......... .......... .......... .......... .......... 2% 108M 43s 2024-03-29T04:10:36.653 INFO:teuthology.orchestra.run.smithi181.stderr: 4000K .......... .......... .......... .......... .......... 2% 110M 43s 2024-03-29T04:10:36.687 INFO:teuthology.orchestra.run.smithi181.stderr: 4050K .......... .......... .......... .......... .......... 2% 1.46M 44s 2024-03-29T04:10:36.687 INFO:teuthology.orchestra.run.smithi181.stderr: 4100K .......... .......... .......... .......... .......... 2% 114M 43s 2024-03-29T04:10:36.687 INFO:teuthology.orchestra.run.smithi181.stderr: 4150K .......... .......... .......... .......... .......... 2% 75.6M 43s 2024-03-29T04:10:36.687 INFO:teuthology.orchestra.run.smithi181.stderr: 4200K .......... .......... .......... .......... .......... 2% 89.2M 42s 2024-03-29T04:10:36.687 INFO:teuthology.orchestra.run.smithi181.stderr: 4250K .......... .......... .......... .......... .......... 2% 110M 42s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4300K .......... .......... .......... .......... .......... 2% 1.46M 42s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4350K .......... .......... .......... .......... .......... 2% 117M 42s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4400K .......... .......... .......... .......... .......... 2% 96.9M 42s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4450K .......... .......... .......... .......... .......... 2% 85.5M 41s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4500K .......... .......... .......... .......... .......... 2% 101M 41s 2024-03-29T04:10:36.724 INFO:teuthology.orchestra.run.smithi181.stderr: 4550K .......... .......... .......... .......... .......... 2% 108M 40s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4600K .......... .......... .......... .......... .......... 2% 1.47M 41s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4650K .......... .......... .......... .......... .......... 2% 111M 40s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4700K .......... .......... .......... .......... .......... 2% 108M 40s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4750K .......... .......... .......... .......... .......... 2% 85.7M 40s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4800K .......... .......... .......... .......... .......... 2% 105M 39s 2024-03-29T04:10:36.760 INFO:teuthology.orchestra.run.smithi181.stderr: 4850K .......... .......... .......... .......... .......... 2% 103M 39s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 4900K .......... .......... .......... .......... .......... 2% 1.47M 40s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 4950K .......... .......... .......... .......... .......... 2% 110M 39s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 5000K .......... .......... .......... .......... .......... 2% 106M 39s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 5050K .......... .......... .......... .......... .......... 3% 112M 38s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 5100K .......... .......... .......... .......... .......... 3% 109M 38s 2024-03-29T04:10:36.796 INFO:teuthology.orchestra.run.smithi181.stderr: 5150K .......... .......... .......... .......... .......... 3% 112M 38s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5200K .......... .......... .......... .......... .......... 3% 1.47M 38s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5250K .......... .......... .......... .......... .......... 3% 113M 38s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5300K .......... .......... .......... .......... .......... 3% 112M 38s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5350K .......... .......... .......... .......... .......... 3% 108M 37s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5400K .......... .......... .......... .......... .......... 3% 110M 37s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5450K .......... .......... .......... .......... .......... 3% 110M 37s 2024-03-29T04:10:36.834 INFO:teuthology.orchestra.run.smithi181.stderr: 5500K .......... .......... .......... .......... .......... 3% 114M 36s 2024-03-29T04:10:36.869 INFO:teuthology.orchestra.run.smithi181.stderr: 5550K .......... .......... .......... .......... .......... 3% 1.48M 37s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5600K .......... .......... .......... .......... .......... 3% 94.8M 37s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5650K .......... .......... .......... .......... .......... 3% 113M 36s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5700K .......... .......... .......... .......... .......... 3% 111M 36s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5750K .......... .......... .......... .......... .......... 3% 111M 36s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5800K .......... .......... .......... .......... .......... 3% 105M 35s 2024-03-29T04:10:36.870 INFO:teuthology.orchestra.run.smithi181.stderr: 5850K .......... .......... .......... .......... .......... 3% 119M 35s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 5900K .......... .......... .......... .......... .......... 3% 1.49M 36s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 5950K .......... .......... .......... .......... .......... 3% 105M 35s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 6000K .......... .......... .......... .......... .......... 3% 111M 35s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 6050K .......... .......... .......... .......... .......... 3% 110M 35s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 6100K .......... .......... .......... .......... .......... 3% 108M 34s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 6150K .......... .......... .......... .......... .......... 3% 110M 34s 2024-03-29T04:10:36.905 INFO:teuthology.orchestra.run.smithi181.stderr: 6200K .......... .......... .......... .......... .......... 3% 108M 34s 2024-03-29T04:10:36.942 INFO:teuthology.orchestra.run.smithi181.stderr: 6250K .......... .......... .......... .......... .......... 3% 1.50M 34s 2024-03-29T04:10:36.942 INFO:teuthology.orchestra.run.smithi181.stderr: 6300K .......... .......... .......... .......... .......... 3% 61.3M 34s 2024-03-29T04:10:36.942 INFO:teuthology.orchestra.run.smithi181.stderr: 6350K .......... .......... .......... .......... .......... 3% 111M 34s 2024-03-29T04:10:36.942 INFO:teuthology.orchestra.run.smithi181.stderr: 6400K .......... .......... .......... .......... .......... 3% 109M 34s 2024-03-29T04:10:36.943 INFO:teuthology.orchestra.run.smithi181.stderr: 6450K .......... .......... .......... .......... .......... 3% 108M 33s 2024-03-29T04:10:36.943 INFO:teuthology.orchestra.run.smithi181.stderr: 6500K .......... .......... .......... .......... .......... 3% 113M 33s 2024-03-29T04:10:36.943 INFO:teuthology.orchestra.run.smithi181.stderr: 6550K .......... .......... .......... .......... .......... 3% 109M 33s 2024-03-29T04:10:36.943 INFO:teuthology.orchestra.run.smithi181.stderr: 6600K .......... .......... .......... .......... .......... 3% 110M 33s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6650K .......... .......... .......... .......... .......... 3% 1.52M 33s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6700K .......... .......... .......... .......... .......... 4% 53.3M 33s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6750K .......... .......... .......... .......... .......... 4% 113M 33s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6800K .......... .......... .......... .......... .......... 4% 113M 32s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6850K .......... .......... .......... .......... .......... 4% 107M 32s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6900K .......... .......... .......... .......... .......... 4% 111M 32s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 6950K .......... .......... .......... .......... .......... 4% 108M 32s 2024-03-29T04:10:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: 7000K .......... .......... .......... .......... .......... 4% 115M 32s 2024-03-29T04:10:37.012 INFO:teuthology.orchestra.run.smithi181.stderr: 7050K .......... .......... .......... .......... .......... 4% 1.53M 32s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7100K .......... .......... .......... .......... .......... 4% 50.7M 32s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7150K .......... .......... .......... .......... .......... 4% 112M 32s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7200K .......... .......... .......... .......... .......... 4% 115M 31s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7250K .......... .......... .......... .......... .......... 4% 111M 31s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7300K .......... .......... .......... .......... .......... 4% 109M 31s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7350K .......... .......... .......... .......... .......... 4% 111M 31s 2024-03-29T04:10:37.013 INFO:teuthology.orchestra.run.smithi181.stderr: 7400K .......... .......... .......... .......... .......... 4% 111M 31s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7450K .......... .......... .......... .......... .......... 4% 1.53M 31s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7500K .......... .......... .......... .......... .......... 4% 50.3M 31s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7550K .......... .......... .......... .......... .......... 4% 107M 31s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7600K .......... .......... .......... .......... .......... 4% 113M 30s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7650K .......... .......... .......... .......... .......... 4% 96.4M 30s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7700K .......... .......... .......... .......... .......... 4% 92.6M 30s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7750K .......... .......... .......... .......... .......... 4% 86.5M 30s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7800K .......... .......... .......... .......... .......... 4% 112M 30s 2024-03-29T04:10:37.051 INFO:teuthology.orchestra.run.smithi181.stderr: 7850K .......... .......... .......... .......... .......... 4% 103M 29s 2024-03-29T04:10:37.086 INFO:teuthology.orchestra.run.smithi181.stderr: 7900K .......... .......... .......... .......... .......... 4% 1.56M 30s 2024-03-29T04:10:37.086 INFO:teuthology.orchestra.run.smithi181.stderr: 7950K .......... .......... .......... .......... .......... 4% 46.9M 30s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8000K .......... .......... .......... .......... .......... 4% 110M 30s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8050K .......... .......... .......... .......... .......... 4% 113M 29s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8100K .......... .......... .......... .......... .......... 4% 111M 29s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8150K .......... .......... .......... .......... .......... 4% 109M 29s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8200K .......... .......... .......... .......... .......... 4% 104M 29s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8250K .......... .......... .......... .......... .......... 4% 110M 29s 2024-03-29T04:10:37.087 INFO:teuthology.orchestra.run.smithi181.stderr: 8300K .......... .......... .......... .......... .......... 4% 82.8M 28s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8350K .......... .......... .......... .......... .......... 4% 1.56M 29s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8400K .......... .......... .......... .......... .......... 5% 57.3M 29s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8450K .......... .......... .......... .......... .......... 5% 115M 29s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8500K .......... .......... .......... .......... .......... 5% 110M 28s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8550K .......... .......... .......... .......... .......... 5% 109M 28s 2024-03-29T04:10:37.122 INFO:teuthology.orchestra.run.smithi181.stderr: 8600K .......... .......... .......... .......... .......... 5% 112M 28s 2024-03-29T04:10:37.123 INFO:teuthology.orchestra.run.smithi181.stderr: 8650K .......... .......... .......... .......... .......... 5% 108M 28s 2024-03-29T04:10:37.123 INFO:teuthology.orchestra.run.smithi181.stderr: 8700K .......... .......... .......... .......... .......... 5% 109M 28s 2024-03-29T04:10:37.123 INFO:teuthology.orchestra.run.smithi181.stderr: 8750K .......... .......... .......... .......... .......... 5% 111M 28s 2024-03-29T04:10:37.123 INFO:teuthology.orchestra.run.smithi181.stderr: 8800K .......... .......... .......... .......... .......... 5% 33.7M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 8850K .......... .......... .......... .......... .......... 5% 1.61M 28s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 8900K .......... .......... .......... .......... .......... 5% 53.2M 28s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 8950K .......... .......... .......... .......... .......... 5% 100M 28s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9000K .......... .......... .......... .......... .......... 5% 99.6M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9050K .......... .......... .......... .......... .......... 5% 112M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9100K .......... .......... .......... .......... .......... 5% 111M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9150K .......... .......... .......... .......... .......... 5% 112M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9200K .......... .......... .......... .......... .......... 5% 112M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9250K .......... .......... .......... .......... .......... 5% 97.0M 27s 2024-03-29T04:10:37.157 INFO:teuthology.orchestra.run.smithi181.stderr: 9300K .......... .......... .......... .......... .......... 5% 29.1M 27s 2024-03-29T04:10:37.173 INFO:teuthology.orchestra.run.smithi181.stderr: 9350K .......... .......... .......... .......... .......... 5% 1.64M 27s 2024-03-29T04:10:37.195 INFO:teuthology.orchestra.run.smithi181.stderr: 9400K .......... .......... .......... .......... .......... 5% 51.6M 27s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9450K .......... .......... .......... .......... .......... 5% 106M 27s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9500K .......... .......... .......... .......... .......... 5% 114M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9550K .......... .......... .......... .......... .......... 5% 108M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9600K .......... .......... .......... .......... .......... 5% 114M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9650K .......... .......... .......... .......... .......... 5% 103M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9700K .......... .......... .......... .......... .......... 5% 111M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9750K .......... .......... .......... .......... .......... 5% 111M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9800K .......... .......... .......... .......... .......... 5% 35.1M 26s 2024-03-29T04:10:37.196 INFO:teuthology.orchestra.run.smithi181.stderr: 9850K .......... .......... .......... .......... .......... 5% 24.8M 26s 2024-03-29T04:10:37.233 INFO:teuthology.orchestra.run.smithi181.stderr: 9900K .......... .......... .......... .......... .......... 5% 1.70M 26s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 9950K .......... .......... .......... .......... .......... 5% 115M 26s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10000K .......... .......... .......... .......... .......... 5% 110M 26s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10050K .......... .......... .......... .......... .......... 5% 108M 26s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10100K .......... .......... .......... .......... .......... 6% 108M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10150K .......... .......... .......... .......... .......... 6% 109M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10200K .......... .......... .......... .......... .......... 6% 105M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10250K .......... .......... .......... .......... .......... 6% 115M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10300K .......... .......... .......... .......... .......... 6% 69.2M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10350K .......... .......... .......... .......... .......... 6% 25.1M 25s 2024-03-29T04:10:37.234 INFO:teuthology.orchestra.run.smithi181.stderr: 10400K .......... .......... .......... .......... .......... 6% 32.3M 25s 2024-03-29T04:10:37.244 INFO:teuthology.orchestra.run.smithi181.stderr: 10450K .......... .......... .......... .......... .......... 6% 1.77M 25s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10500K .......... .......... .......... .......... .......... 6% 56.8M 25s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10550K .......... .......... .......... .......... .......... 6% 105M 25s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10600K .......... .......... .......... .......... .......... 6% 104M 25s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10650K .......... .......... .......... .......... .......... 6% 111M 25s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10700K .......... .......... .......... .......... .......... 6% 98.3M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10750K .......... .......... .......... .......... .......... 6% 115M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10800K .......... .......... .......... .......... .......... 6% 104M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10850K .......... .......... .......... .......... .......... 6% 67.0M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10900K .......... .......... .......... .......... .......... 6% 36.0M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 10950K .......... .......... .......... .......... .......... 6% 24.4M 24s 2024-03-29T04:10:37.272 INFO:teuthology.orchestra.run.smithi181.stderr: 11000K .......... .......... .......... .......... .......... 6% 32.6M 24s 2024-03-29T04:10:37.306 INFO:teuthology.orchestra.run.smithi181.stderr: 11050K .......... .......... .......... .......... .......... 6% 1.82M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11100K .......... .......... .......... .......... .......... 6% 114M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11150K .......... .......... .......... .......... .......... 6% 111M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11200K .......... .......... .......... .......... .......... 6% 114M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11250K .......... .......... .......... .......... .......... 6% 112M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11300K .......... .......... .......... .......... .......... 6% 110M 24s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11350K .......... .......... .......... .......... .......... 6% 107M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11400K .......... .......... .......... .......... .......... 6% 114M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11450K .......... .......... .......... .......... .......... 6% 49.9M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11500K .......... .......... .......... .......... .......... 6% 32.8M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11550K .......... .......... .......... .......... .......... 6% 24.4M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11600K .......... .......... .......... .......... .......... 6% 31.5M 23s 2024-03-29T04:10:37.307 INFO:teuthology.orchestra.run.smithi181.stderr: 11650K .......... .......... .......... .......... .......... 6% 36.6M 23s 2024-03-29T04:10:37.317 INFO:teuthology.orchestra.run.smithi181.stderr: 11700K .......... .......... .......... .......... .......... 6% 1.92M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 11750K .......... .......... .......... .......... .......... 7% 12.5M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 11800K .......... .......... .......... .......... .......... 7% 104M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 11850K .......... .......... .......... .......... .......... 7% 482M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 11900K .......... .......... .......... .......... .......... 7% 267M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 11950K .......... .......... .......... .......... .......... 7% 329M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12000K .......... .......... .......... .......... .......... 7% 401M 23s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12050K .......... .......... .......... .......... .......... 7% 482M 22s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12100K .......... .......... .......... .......... .......... 7% 291M 22s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12150K .......... .......... .......... .......... .......... 7% 47.2M 22s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12200K .......... .......... .......... .......... .......... 7% 25.5M 22s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12250K .......... .......... .......... .......... .......... 7% 32.6M 22s 2024-03-29T04:10:37.333 INFO:teuthology.orchestra.run.smithi181.stderr: 12300K .......... .......... .......... .......... .......... 7% 24.6M 22s 2024-03-29T04:10:37.354 INFO:teuthology.orchestra.run.smithi181.stderr: 12350K .......... .......... .......... .......... .......... 7% 1.85M 22s 2024-03-29T04:10:37.390 INFO:teuthology.orchestra.run.smithi181.stderr: 12400K .......... .......... .......... .......... .......... 7% 19.6M 22s 2024-03-29T04:10:37.390 INFO:teuthology.orchestra.run.smithi181.stderr: 12450K .......... .......... .......... .......... .......... 7% 112M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12500K .......... .......... .......... .......... .......... 7% 113M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12550K .......... .......... .......... .......... .......... 7% 112M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12600K .......... .......... .......... .......... .......... 7% 107M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12650K .......... .......... .......... .......... .......... 7% 112M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12700K .......... .......... .......... .......... .......... 7% 108M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12750K .......... .......... .......... .......... .......... 7% 95.0M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12800K .......... .......... .......... .......... .......... 7% 26.0M 22s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12850K .......... .......... .......... .......... .......... 7% 31.8M 21s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12900K .......... .......... .......... .......... .......... 7% 35.0M 21s 2024-03-29T04:10:37.391 INFO:teuthology.orchestra.run.smithi181.stderr: 12950K .......... .......... .......... .......... .......... 7% 24.4M 21s 2024-03-29T04:10:37.392 INFO:teuthology.orchestra.run.smithi181.stderr: 13000K .......... .......... .......... .......... .......... 7% 32.9M 21s 2024-03-29T04:10:40.261 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:41.842 INFO:teuthology.orchestra.run.smithi181.stderr: 13050K .......... .......... .......... .......... .......... 7% 2.05M 21s 2024-03-29T04:10:41.842 INFO:teuthology.orchestra.run.smithi181.stderr: 13100K .......... .......... .......... .......... .......... 7% 110M 21s 2024-03-29T04:10:41.842 INFO:teuthology.orchestra.run.smithi181.stderr: 13150K .......... .......... .......... .......... .......... 7% 111M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13200K .......... .......... .......... .......... .......... 7% 112M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13250K .......... .......... .......... .......... .......... 7% 109M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13300K .......... .......... .......... .......... .......... 7% 105M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13350K .......... .......... .......... .......... .......... 7% 112M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13400K .......... .......... .......... .......... .......... 7% 113M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13450K .......... .......... .......... .......... .......... 8% 37.7M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13500K .......... .......... .......... .......... .......... 8% 33.3M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13550K .......... .......... .......... .......... .......... 8% 24.4M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13600K .......... .......... .......... .......... .......... 8% 33.5M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13650K .......... .......... .......... .......... .......... 8% 33.6M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13700K .......... .......... .......... .......... .......... 8% 24.5M 20s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13750K .......... .......... .......... .......... .......... 8% 2.37M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13800K .......... .......... .......... .......... .......... 8% 20.0M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13850K .......... .......... .......... .......... .......... 8% 110M 21s 2024-03-29T04:10:41.843 INFO:teuthology.orchestra.run.smithi181.stderr: 13900K .......... .......... .......... .......... .......... 8% 82.8M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 13950K .......... .......... .......... .......... .......... 8% 113M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14000K .......... .......... .......... .......... .......... 8% 107M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14050K .......... .......... .......... .......... .......... 8% 110M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14100K .......... .......... .......... .......... .......... 8% 112M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14150K .......... .......... .......... .......... .......... 8% 50.4M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14200K .......... .......... .......... .......... .......... 8% 35.9M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14250K .......... .......... .......... .......... .......... 8% 29.8M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14300K .......... .......... .......... .......... .......... 8% 22.9M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14350K .......... .......... .......... .......... .......... 8% 34.2M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14400K .......... .......... .......... .......... .......... 8% 24.7M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14450K .......... .......... .......... .......... .......... 8% 35.5M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14500K .......... .......... .......... .......... .......... 8% 2.27M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14550K .......... .......... .......... .......... .......... 8% 117M 20s 2024-03-29T04:10:41.844 INFO:teuthology.orchestra.run.smithi181.stderr: 14600K .......... .......... .......... .......... .......... 8% 110M 20s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14650K .......... .......... .......... .......... .......... 8% 116M 20s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14700K .......... .......... .......... .......... .......... 8% 101M 20s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14750K .......... .......... .......... .......... .......... 8% 115M 20s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14800K .......... .......... .......... .......... .......... 8% 108M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14850K .......... .......... .......... .......... .......... 8% 109M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14900K .......... .......... .......... .......... .......... 8% 42.3M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 14950K .......... .......... .......... .......... .......... 8% 36.9M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15000K .......... .......... .......... .......... .......... 8% 28.3M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15050K .......... .......... .......... .......... .......... 8% 24.6M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15100K .......... .......... .......... .......... .......... 8% 35.2M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15150K .......... .......... .......... .......... .......... 9% 24.6M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15200K .......... .......... .......... .......... .......... 9% 38.3M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15250K .......... .......... .......... .......... .......... 9% 28.6M 19s 2024-03-29T04:10:41.845 INFO:teuthology.orchestra.run.smithi181.stderr: 15300K .......... .......... .......... .......... .......... 9% 2.44M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15350K .......... .......... .......... .......... .......... 9% 114M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15400K .......... .......... .......... .......... .......... 9% 110M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15450K .......... .......... .......... .......... .......... 9% 111M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15500K .......... .......... .......... .......... .......... 9% 111M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15550K .......... .......... .......... .......... .......... 9% 107M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15600K .......... .......... .......... .......... .......... 9% 112M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15650K .......... .......... .......... .......... .......... 9% 109M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15700K .......... .......... .......... .......... .......... 9% 56.1M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15750K .......... .......... .......... .......... .......... 9% 26.3M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15800K .......... .......... .......... .......... .......... 9% 23.3M 19s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15850K .......... .......... .......... .......... .......... 9% 37.1M 18s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15900K .......... .......... .......... .......... .......... 9% 25.9M 18s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 15950K .......... .......... .......... .......... .......... 9% 36.2M 18s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 16000K .......... .......... .......... .......... .......... 9% 27.5M 18s 2024-03-29T04:10:41.846 INFO:teuthology.orchestra.run.smithi181.stderr: 16050K .......... .......... .......... .......... .......... 9% 23.6M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16100K .......... .......... .......... .......... .......... 9% 3.47M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16150K .......... .......... .......... .......... .......... 9% 9.17M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16200K .......... .......... .......... .......... .......... 9% 114M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16250K .......... .......... .......... .......... .......... 9% 109M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16300K .......... .......... .......... .......... .......... 9% 111M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16350K .......... .......... .......... .......... .......... 9% 113M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16400K .......... .......... .......... .......... .......... 9% 108M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16450K .......... .......... .......... .......... .......... 9% 110M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16500K .......... .......... .......... .......... .......... 9% 114M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16550K .......... .......... .......... .......... .......... 9% 29.9M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16600K .......... .......... .......... .......... .......... 9% 42.2M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16650K .......... .......... .......... .......... .......... 9% 23.1M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16700K .......... .......... .......... .......... .......... 9% 41.9M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16750K .......... .......... .......... .......... .......... 9% 25.2M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16800K .......... .......... .......... .......... .......... 9% 24.4M 18s 2024-03-29T04:10:41.847 INFO:teuthology.orchestra.run.smithi181.stderr: 16850K .......... .......... .......... .......... .......... 10% 37.6M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 16900K .......... .......... .......... .......... .......... 10% 24.2M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 16950K .......... .......... .......... .......... .......... 10% 44.6M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17000K .......... .......... .......... .......... .......... 10% 2.74M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17050K .......... .......... .......... .......... .......... 10% 94.2M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17100K .......... .......... .......... .......... .......... 10% 110M 18s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17150K .......... .......... .......... .......... .......... 10% 101M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17200K .......... .......... .......... .......... .......... 10% 46.2M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17250K .......... .......... .......... .......... .......... 10% 103M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17300K .......... .......... .......... .......... .......... 10% 112M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17350K .......... .......... .......... .......... .......... 10% 115M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17400K .......... .......... .......... .......... .......... 10% 85.4M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17450K .......... .......... .......... .......... .......... 10% 43.2M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17500K .......... .......... .......... .......... .......... 10% 25.6M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17550K .......... .......... .......... .......... .......... 10% 25.4M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17600K .......... .......... .......... .......... .......... 10% 34.1M 17s 2024-03-29T04:10:41.848 INFO:teuthology.orchestra.run.smithi181.stderr: 17650K .......... .......... .......... .......... .......... 10% 24.6M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17700K .......... .......... .......... .......... .......... 10% 41.8M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17750K .......... .......... .......... .......... .......... 10% 24.3M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17800K .......... .......... .......... .......... .......... 10% 32.2M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17850K .......... .......... .......... .......... .......... 10% 28.7M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17900K .......... .......... .......... .......... .......... 10% 2.99M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 17950K .......... .......... .......... .......... .......... 10% 105M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18000K .......... .......... .......... .......... .......... 10% 112M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18050K .......... .......... .......... .......... .......... 10% 109M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18100K .......... .......... .......... .......... .......... 10% 46.6M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18150K .......... .......... .......... .......... .......... 10% 112M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18200K .......... .......... .......... .......... .......... 10% 108M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18250K .......... .......... .......... .......... .......... 10% 111M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18300K .......... .......... .......... .......... .......... 10% 64.7M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18350K .......... .......... .......... .......... .......... 10% 38.6M 17s 2024-03-29T04:10:41.849 INFO:teuthology.orchestra.run.smithi181.stderr: 18400K .......... .......... .......... .......... .......... 10% 30.0M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18450K .......... .......... .......... .......... .......... 10% 33.8M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18500K .......... .......... .......... .......... .......... 11% 24.0M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18550K .......... .......... .......... .......... .......... 11% 28.0M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18600K .......... .......... .......... .......... .......... 11% 24.0M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18650K .......... .......... .......... .......... .......... 11% 39.5M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18700K .......... .......... .......... .......... .......... 11% 37.3M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18750K .......... .......... .......... .......... .......... 11% 23.8M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18800K .......... .......... .......... .......... .......... 11% 4.36M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18850K .......... .......... .......... .......... .......... 11% 15.9M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18900K .......... .......... .......... .......... .......... 11% 20.2M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 18950K .......... .......... .......... .......... .......... 11% 106M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 19000K .......... .......... .......... .......... .......... 11% 113M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 19050K .......... .......... .......... .......... .......... 11% 48.8M 16s 2024-03-29T04:10:41.850 INFO:teuthology.orchestra.run.smithi181.stderr: 19100K .......... .......... .......... .......... .......... 11% 112M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19150K .......... .......... .......... .......... .......... 11% 97.8M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19200K .......... .......... .......... .......... .......... 11% 121M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19250K .......... .......... .......... .......... .......... 11% 108M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19300K .......... .......... .......... .......... .......... 11% 30.2M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19350K .......... .......... .......... .......... .......... 11% 37.5M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19400K .......... .......... .......... .......... .......... 11% 24.9M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19450K .......... .......... .......... .......... .......... 11% 32.7M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19500K .......... .......... .......... .......... .......... 11% 31.2M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19550K .......... .......... .......... .......... .......... 11% 25.2M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19600K .......... .......... .......... .......... .......... 11% 22.4M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19650K .......... .......... .......... .......... .......... 11% 38.0M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19700K .......... .......... .......... .......... .......... 11% 37.3M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19750K .......... .......... .......... .......... .......... 11% 26.5M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19800K .......... .......... .......... .......... .......... 11% 4.92M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19850K .......... .......... .......... .......... .......... 11% 9.56M 16s 2024-03-29T04:10:41.851 INFO:teuthology.orchestra.run.smithi181.stderr: 19900K .......... .......... .......... .......... .......... 11% 113M 16s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 19950K .......... .......... .......... .......... .......... 11% 110M 16s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20000K .......... .......... .......... .......... .......... 11% 111M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20050K .......... .......... .......... .......... .......... 11% 51.4M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20100K .......... .......... .......... .......... .......... 11% 114M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20150K .......... .......... .......... .......... .......... 11% 102M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20200K .......... .......... .......... .......... .......... 12% 109M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20250K .......... .......... .......... .......... .......... 12% 113M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20300K .......... .......... .......... .......... .......... 12% 25.1M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20350K .......... .......... .......... .......... .......... 12% 36.6M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20400K .......... .......... .......... .......... .......... 12% 32.8M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20450K .......... .......... .......... .......... .......... 12% 26.4M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20500K .......... .......... .......... .......... .......... 12% 30.7M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20550K .......... .......... .......... .......... .......... 12% 29.9M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20600K .......... .......... .......... .......... .......... 12% 20.4M 15s 2024-03-29T04:10:41.852 INFO:teuthology.orchestra.run.smithi181.stderr: 20650K .......... .......... .......... .......... .......... 12% 47.9M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20700K .......... .......... .......... .......... .......... 12% 21.1M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20750K .......... .......... .......... .......... .......... 12% 39.0M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20800K .......... .......... .......... .......... .......... 12% 31.4M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20850K .......... .......... .......... .......... .......... 12% 5.89M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20900K .......... .......... .......... .......... .......... 12% 10.2M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 20950K .......... .......... .......... .......... .......... 12% 83.4M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21000K .......... .......... .......... .......... .......... 12% 112M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21050K .......... .......... .......... .......... .......... 12% 112M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21100K .......... .......... .......... .......... .......... 12% 108M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21150K .......... .......... .......... .......... .......... 12% 62.2M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21200K .......... .......... .......... .......... .......... 12% 110M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21250K .......... .......... .......... .......... .......... 12% 111M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21300K .......... .......... .......... .......... .......... 12% 66.6M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21350K .......... .......... .......... .......... .......... 12% 23.3M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21400K .......... .......... .......... .......... .......... 12% 38.7M 15s 2024-03-29T04:10:41.853 INFO:teuthology.orchestra.run.smithi181.stderr: 21450K .......... .......... .......... .......... .......... 12% 25.2M 15s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21500K .......... .......... .......... .......... .......... 12% 55.4M 15s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21550K .......... .......... .......... .......... .......... 12% 24.4M 15s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21600K .......... .......... .......... .......... .......... 12% 29.9M 15s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21650K .......... .......... .......... .......... .......... 12% 24.2M 15s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21700K .......... .......... .......... .......... .......... 12% 43.2M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21750K .......... .......... .......... .......... .......... 12% 26.7M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21800K .......... .......... .......... .......... .......... 12% 25.8M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21850K .......... .......... .......... .......... .......... 12% 32.2M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21900K .......... .......... .......... .......... .......... 13% 24.1M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 21950K .......... .......... .......... .......... .......... 13% 7.76M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 22000K .......... .......... .......... .......... .......... 13% 17.9M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 22050K .......... .......... .......... .......... .......... 13% 20.1M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 22100K .......... .......... .......... .......... .......... 13% 88.3M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 22150K .......... .......... .......... .......... .......... 13% 106M 14s 2024-03-29T04:10:41.854 INFO:teuthology.orchestra.run.smithi181.stderr: 22200K .......... .......... .......... .......... .......... 13% 114M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22250K .......... .......... .......... .......... .......... 13% 84.4M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22300K .......... .......... .......... .......... .......... 13% 76.3M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22350K .......... .......... .......... .......... .......... 13% 65.6M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22400K .......... .......... .......... .......... .......... 13% 24.5M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22450K .......... .......... .......... .......... .......... 13% 24.4M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22500K .......... .......... .......... .......... .......... 13% 60.4M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22550K .......... .......... .......... .......... .......... 13% 23.0M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22600K .......... .......... .......... .......... .......... 13% 24.2M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22650K .......... .......... .......... .......... .......... 13% 44.4M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22700K .......... .......... .......... .......... .......... 13% 23.0M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22750K .......... .......... .......... .......... .......... 13% 40.6M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22800K .......... .......... .......... .......... .......... 13% 25.2M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22850K .......... .......... .......... .......... .......... 13% 34.7M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22900K .......... .......... .......... .......... .......... 13% 25.0M 14s 2024-03-29T04:10:41.855 INFO:teuthology.orchestra.run.smithi181.stderr: 22950K .......... .......... .......... .......... .......... 13% 23.9M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23000K .......... .......... .......... .......... .......... 13% 53.7M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23050K .......... .......... .......... .......... .......... 13% 24.7M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23100K .......... .......... .......... .......... .......... 13% 14.0M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23150K .......... .......... .......... .......... .......... 13% 18.8M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23200K .......... .......... .......... .......... .......... 13% 19.9M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23250K .......... .......... .......... .......... .......... 13% 101M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23300K .......... .......... .......... .......... .......... 13% 85.5M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23350K .......... .......... .......... .......... .......... 13% 100M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23400K .......... .......... .......... .......... .......... 13% 26.6M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23450K .......... .......... .......... .......... .......... 13% 24.8M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23500K .......... .......... .......... .......... .......... 13% 51.1M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23550K .......... .......... .......... .......... .......... 14% 24.7M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23600K .......... .......... .......... .......... .......... 14% 27.2M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23650K .......... .......... .......... .......... .......... 14% 25.6M 14s 2024-03-29T04:10:41.856 INFO:teuthology.orchestra.run.smithi181.stderr: 23700K .......... .......... .......... .......... .......... 14% 53.4M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 23750K .......... .......... .......... .......... .......... 14% 27.8M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 23800K .......... .......... .......... .......... .......... 14% 24.3M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 23850K .......... .......... .......... .......... .......... 14% 32.2M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 23900K .......... .......... .......... .......... .......... 14% 25.1M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 23950K .......... .......... .......... .......... .......... 14% 29.1M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24000K .......... .......... .......... .......... .......... 14% 34.5M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24050K .......... .......... .......... .......... .......... 14% 25.1M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24100K .......... .......... .......... .......... .......... 14% 25.1M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24150K .......... .......... .......... .......... .......... 14% 34.0M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24200K .......... .......... .......... .......... .......... 14% 24.8M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24250K .......... .......... .......... .......... .......... 14% 50.7M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24300K .......... .......... .......... .......... .......... 14% 23.7M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24350K .......... .......... .......... .......... .......... 14% 30.5M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24400K .......... .......... .......... .......... .......... 14% 23.2M 13s 2024-03-29T04:10:41.857 INFO:teuthology.orchestra.run.smithi181.stderr: 24450K .......... .......... .......... .......... .......... 14% 26.8M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24500K .......... .......... .......... .......... .......... 14% 24.1M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24550K .......... .......... .......... .......... .......... 14% 59.7M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24600K .......... .......... .......... .......... .......... 14% 31.5M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24650K .......... .......... .......... .......... .......... 14% 24.3M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24700K .......... .......... .......... .......... .......... 14% 25.5M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24750K .......... .......... .......... .......... .......... 14% 25.3M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24800K .......... .......... .......... .......... .......... 14% 49.3M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24850K .......... .......... .......... .......... .......... 14% 24.2M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24900K .......... .......... .......... .......... .......... 14% 25.0M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 24950K .......... .......... .......... .......... .......... 14% 43.3M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 25000K .......... .......... .......... .......... .......... 14% 30.8M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 25050K .......... .......... .......... .......... .......... 14% 31.9M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 25100K .......... .......... .......... .......... .......... 14% 23.2M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 25150K .......... .......... .......... .......... .......... 14% 24.0M 13s 2024-03-29T04:10:41.858 INFO:teuthology.orchestra.run.smithi181.stderr: 25200K .......... .......... .......... .......... .......... 14% 34.7M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25250K .......... .......... .......... .......... .......... 15% 24.8M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25300K .......... .......... .......... .......... .......... 15% 49.0M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25350K .......... .......... .......... .......... .......... 15% 24.2M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25400K .......... .......... .......... .......... .......... 15% 38.1M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25450K .......... .......... .......... .......... .......... 15% 22.7M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25500K .......... .......... .......... .......... .......... 15% 24.8M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25550K .......... .......... .......... .......... .......... 15% 52.1M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25600K .......... .......... .......... .......... .......... 15% 33.0M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25650K .......... .......... .......... .......... .......... 15% 24.0M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25700K .......... .......... .......... .......... .......... 15% 24.4M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25750K .......... .......... .......... .......... .......... 15% 24.9M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25800K .......... .......... .......... .......... .......... 15% 70.2M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25850K .......... .......... .......... .......... .......... 15% 23.2M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25900K .......... .......... .......... .......... .......... 15% 32.1M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 25950K .......... .......... .......... .......... .......... 15% 24.7M 13s 2024-03-29T04:10:41.859 INFO:teuthology.orchestra.run.smithi181.stderr: 26000K .......... .......... .......... .......... .......... 15% 24.4M 13s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26050K .......... .......... .......... .......... .......... 15% 50.6M 13s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26100K .......... .......... .......... .......... .......... 15% 24.4M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26150K .......... .......... .......... .......... .......... 15% 23.9M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26200K .......... .......... .......... .......... .......... 15% 37.3M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26250K .......... .......... .......... .......... .......... 15% 26.1M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26300K .......... .......... .......... .......... .......... 15% 45.0M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26350K .......... .......... .......... .......... .......... 15% 23.7M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26400K .......... .......... .......... .......... .......... 15% 34.9M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26450K .......... .......... .......... .......... .......... 15% 24.5M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26500K .......... .......... .......... .......... .......... 15% 29.9M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26550K .......... .......... .......... .......... .......... 15% 34.5M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26600K .......... .......... .......... .......... .......... 15% 24.9M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26650K .......... .......... .......... .......... .......... 15% 38.1M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26700K .......... .......... .......... .......... .......... 15% 24.7M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26750K .......... .......... .......... .......... .......... 15% 28.7M 12s 2024-03-29T04:10:41.860 INFO:teuthology.orchestra.run.smithi181.stderr: 26800K .......... .......... .......... .......... .......... 15% 35.2M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 26850K .......... .......... .......... .......... .......... 15% 25.1M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 26900K .......... .......... .......... .......... .......... 15% 33.2M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 26950K .......... .......... .......... .......... .......... 16% 24.2M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27000K .......... .......... .......... .......... .......... 16% 24.6M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27050K .......... .......... .......... .......... .......... 16% 58.4M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27100K .......... .......... .......... .......... .......... 16% 27.1M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27150K .......... .......... .......... .......... .......... 16% 27.9M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27200K .......... .......... .......... .......... .......... 16% 24.7M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27250K .......... .......... .......... .......... .......... 16% 24.5M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27300K .......... .......... .......... .......... .......... 16% 52.7M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27350K .......... .......... .......... .......... .......... 16% 23.7M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27400K .......... .......... .......... .......... .......... 16% 34.8M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27450K .......... .......... .......... .......... .......... 16% 22.7M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27500K .......... .......... .......... .......... .......... 16% 26.6M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27550K .......... .......... .......... .......... .......... 16% 50.2M 12s 2024-03-29T04:10:41.861 INFO:teuthology.orchestra.run.smithi181.stderr: 27600K .......... .......... .......... .......... .......... 16% 28.6M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27650K .......... .......... .......... .......... .......... 16% 21.7M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27700K .......... .......... .......... .......... .......... 16% 52.7M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27750K .......... .......... .......... .......... .......... 16% 24.8M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27800K .......... .......... .......... .......... .......... 16% 32.7M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27850K .......... .......... .......... .......... .......... 16% 24.3M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27900K .......... .......... .......... .......... .......... 16% 26.9M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 27950K .......... .......... .......... .......... .......... 16% 31.0M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28000K .......... .......... .......... .......... .......... 16% 24.5M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28050K .......... .......... .......... .......... .......... 16% 42.5M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28100K .......... .......... .......... .......... .......... 16% 25.5M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28150K .......... .......... .......... .......... .......... 16% 24.0M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28200K .......... .......... .......... .......... .......... 16% 32.5M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28250K .......... .......... .......... .......... .......... 16% 25.9M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28300K .......... .......... .......... .......... .......... 16% 52.9M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28350K .......... .......... .......... .......... .......... 16% 29.7M 12s 2024-03-29T04:10:41.862 INFO:teuthology.orchestra.run.smithi181.stderr: 28400K .......... .......... .......... .......... .......... 16% 24.6M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28450K .......... .......... .......... .......... .......... 16% 24.2M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28500K .......... .......... .......... .......... .......... 16% 24.7M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28550K .......... .......... .......... .......... .......... 16% 59.2M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28600K .......... .......... .......... .......... .......... 16% 23.1M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28650K .......... .......... .......... .......... .......... 17% 34.3M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28700K .......... .......... .......... .......... .......... 17% 24.6M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28750K .......... .......... .......... .......... .......... 17% 19.1M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28800K .......... .......... .......... .......... .......... 17% 102M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28850K .......... .......... .......... .......... .......... 17% 27.9M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28900K .......... .......... .......... .......... .......... 17% 22.8M 12s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 28950K .......... .......... .......... .......... .......... 17% 27.3M 11s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 29000K .......... .......... .......... .......... .......... 17% 36.7M 11s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 29050K .......... .......... .......... .......... .......... 17% 37.1M 11s 2024-03-29T04:10:41.863 INFO:teuthology.orchestra.run.smithi181.stderr: 29100K .......... .......... .......... .......... .......... 17% 26.1M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29150K .......... .......... .......... .......... .......... 17% 32.0M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29200K .......... .......... .......... .......... .......... 17% 24.8M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29250K .......... .......... .......... .......... .......... 17% 27.5M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29300K .......... .......... .......... .......... .......... 17% 40.1M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29350K .......... .......... .......... .......... .......... 17% 25.1M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29400K .......... .......... .......... .......... .......... 17% 34.4M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29450K .......... .......... .......... .......... .......... 17% 24.6M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29500K .......... .......... .......... .......... .......... 17% 24.8M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29550K .......... .......... .......... .......... .......... 17% 52.5M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29600K .......... .......... .......... .......... .......... 17% 23.7M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29650K .......... .......... .......... .......... .......... 17% 23.7M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29700K .......... .......... .......... .......... .......... 17% 42.0M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29750K .......... .......... .......... .......... .......... 17% 22.6M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29800K .......... .......... .......... .......... .......... 17% 24.1M 11s 2024-03-29T04:10:41.864 INFO:teuthology.orchestra.run.smithi181.stderr: 29850K .......... .......... .......... .......... .......... 17% 58.7M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 29900K .......... .......... .......... .......... .......... 17% 23.9M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 29950K .......... .......... .......... .......... .......... 17% 32.6M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30000K .......... .......... .......... .......... .......... 17% 24.8M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30050K .......... .......... .......... .......... .......... 17% 47.9M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30100K .......... .......... .......... .......... .......... 17% 25.8M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30150K .......... .......... .......... .......... .......... 17% 26.5M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30200K .......... .......... .......... .......... .......... 17% 25.2M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30250K .......... .......... .......... .......... .......... 17% 36.5M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30300K .......... .......... .......... .......... .......... 18% 21.5M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30350K .......... .......... .......... .......... .......... 18% 47.0M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30400K .......... .......... .......... .......... .......... 18% 34.3M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30450K .......... .......... .......... .......... .......... 18% 23.5M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30500K .......... .......... .......... .......... .......... 18% 26.2M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30550K .......... .......... .......... .......... .......... 18% 21.7M 11s 2024-03-29T04:10:41.865 INFO:teuthology.orchestra.run.smithi181.stderr: 30600K .......... .......... .......... .......... .......... 18% 51.5M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30650K .......... .......... .......... .......... .......... 18% 24.6M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30700K .......... .......... .......... .......... .......... 18% 34.0M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30750K .......... .......... .......... .......... .......... 18% 24.5M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30800K .......... .......... .......... .......... .......... 18% 28.7M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30850K .......... .......... .......... .......... .......... 18% 42.7M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30900K .......... .......... .......... .......... .......... 18% 27.8M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 30950K .......... .......... .......... .......... .......... 18% 27.7M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31000K .......... .......... .......... .......... .......... 18% 23.5M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31050K .......... .......... .......... .......... .......... 18% 24.4M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31100K .......... .......... .......... .......... .......... 18% 70.3M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31150K .......... .......... .......... .......... .......... 18% 21.1M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31200K .......... .......... .......... .......... .......... 18% 25.0M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31250K .......... .......... .......... .......... .......... 18% 30.8M 11s 2024-03-29T04:10:41.866 INFO:teuthology.orchestra.run.smithi181.stderr: 31300K .......... .......... .......... .......... .......... 18% 21.8M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31350K .......... .......... .......... .......... .......... 18% 40.2M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31400K .......... .......... .......... .......... .......... 18% 32.8M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31450K .......... .......... .......... .......... .......... 18% 31.0M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31500K .......... .......... .......... .......... .......... 18% 24.1M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31550K .......... .......... .......... .......... .......... 18% 54.6M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31600K .......... .......... .......... .......... .......... 18% 30.4M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31650K .......... .......... .......... .......... .......... 18% 22.4M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31700K .......... .......... .......... .......... .......... 18% 41.3M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31750K .......... .......... .......... .......... .......... 18% 22.5M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31800K .......... .......... .......... .......... .......... 18% 29.1M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31850K .......... .......... .......... .......... .......... 18% 40.5M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31900K .......... .......... .......... .......... .......... 18% 22.9M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 31950K .......... .......... .......... .......... .......... 18% 39.2M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 32000K .......... .......... .......... .......... .......... 19% 23.2M 11s 2024-03-29T04:10:41.867 INFO:teuthology.orchestra.run.smithi181.stderr: 32050K .......... .......... .......... .......... .......... 19% 24.6M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32100K .......... .......... .......... .......... .......... 19% 57.5M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32150K .......... .......... .......... .......... .......... 19% 23.8M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32200K .......... .......... .......... .......... .......... 19% 31.9M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32250K .......... .......... .......... .......... .......... 19% 24.4M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32300K .......... .......... .......... .......... .......... 19% 25.0M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32350K .......... .......... .......... .......... .......... 19% 39.8M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32400K .......... .......... .......... .......... .......... 19% 23.3M 11s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32450K .......... .......... .......... .......... .......... 19% 28.5M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32500K .......... .......... .......... .......... .......... 19% 21.6M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32550K .......... .......... .......... .......... .......... 19% 22.8M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32600K .......... .......... .......... .......... .......... 19% 38.6M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32650K .......... .......... .......... .......... .......... 19% 68.5M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32700K .......... .......... .......... .......... .......... 19% 30.3M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32750K .......... .......... .......... .......... .......... 19% 23.9M 10s 2024-03-29T04:10:41.868 INFO:teuthology.orchestra.run.smithi181.stderr: 32800K .......... .......... .......... .......... .......... 19% 28.1M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 32850K .......... .......... .......... .......... .......... 19% 49.6M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 32900K .......... .......... .......... .......... .......... 19% 27.6M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 32950K .......... .......... .......... .......... .......... 19% 23.3M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33000K .......... .......... .......... .......... .......... 19% 39.5M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33050K .......... .......... .......... .......... .......... 19% 22.6M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33100K .......... .......... .......... .......... .......... 19% 43.4M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33150K .......... .......... .......... .......... .......... 19% 25.8M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33200K .......... .......... .......... .......... .......... 19% 31.3M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33250K .......... .......... .......... .......... .......... 19% 21.4M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33300K .......... .......... .......... .......... .......... 19% 20.6M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33350K .......... .......... .......... .......... .......... 19% 45.6M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33400K .......... .......... .......... .......... .......... 19% 22.7M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33450K .......... .......... .......... .......... .......... 19% 30.5M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33500K .......... .......... .......... .......... .......... 19% 20.7M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33550K .......... .......... .......... .......... .......... 19% 26.4M 10s 2024-03-29T04:10:41.869 INFO:teuthology.orchestra.run.smithi181.stderr: 33600K .......... .......... .......... .......... .......... 19% 31.0M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33650K .......... .......... .......... .......... .......... 19% 17.7M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33700K .......... .......... .......... .......... .......... 20% 24.4M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33750K .......... .......... .......... .......... .......... 20% 22.9M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33800K .......... .......... .......... .......... .......... 20% 21.2M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33850K .......... .......... .......... .......... .......... 20% 45.3M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33900K .......... .......... .......... .......... .......... 20% 21.3M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 33950K .......... .......... .......... .......... .......... 20% 39.3M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34000K .......... .......... .......... .......... .......... 20% 105M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34050K .......... .......... .......... .......... .......... 20% 113M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34100K .......... .......... .......... .......... .......... 20% 62.6M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34150K .......... .......... .......... .......... .......... 20% 32.0M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34200K .......... .......... .......... .......... .......... 20% 27.3M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34250K .......... .......... .......... .......... .......... 20% 23.0M 10s 2024-03-29T04:10:41.870 INFO:teuthology.orchestra.run.smithi181.stderr: 34300K .......... .......... .......... .......... .......... 20% 27.8M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34350K .......... .......... .......... .......... .......... 20% 37.0M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34400K .......... .......... .......... .......... .......... 20% 21.4M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34450K .......... .......... .......... .......... .......... 20% 35.3M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34500K .......... .......... .......... .......... .......... 20% 23.1M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34550K .......... .......... .......... .......... .......... 20% 20.0M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34600K .......... .......... .......... .......... .......... 20% 36.6M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34650K .......... .......... .......... .......... .......... 20% 21.6M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34700K .......... .......... .......... .......... .......... 20% 22.5M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34750K .......... .......... .......... .......... .......... 20% 25.6M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34800K .......... .......... .......... .......... .......... 20% 23.0M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34850K .......... .......... .......... .......... .......... 20% 37.1M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34900K .......... .......... .......... .......... .......... 20% 22.4M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 34950K .......... .......... .......... .......... .......... 20% 25.6M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 35000K .......... .......... .......... .......... .......... 20% 28.8M 10s 2024-03-29T04:10:41.871 INFO:teuthology.orchestra.run.smithi181.stderr: 35050K .......... .......... .......... .......... .......... 20% 20.7M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35100K .......... .......... .......... .......... .......... 20% 83.5M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35150K .......... .......... .......... .......... .......... 20% 69.2M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35200K .......... .......... .......... .......... .......... 20% 44.9M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35250K .......... .......... .......... .......... .......... 20% 106M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35300K .......... .......... .......... .......... .......... 20% 35.6M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35350K .......... .......... .......... .......... .......... 21% 28.4M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35400K .......... .......... .......... .......... .......... 21% 26.3M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35450K .......... .......... .......... .......... .......... 21% 26.3M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35500K .......... .......... .......... .......... .......... 21% 25.2M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35550K .......... .......... .......... .......... .......... 21% 35.7M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35600K .......... .......... .......... .......... .......... 21% 33.7M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35650K .......... .......... .......... .......... .......... 21% 34.5M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35700K .......... .......... .......... .......... .......... 21% 22.7M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35750K .......... .......... .......... .......... .......... 21% 37.8M 10s 2024-03-29T04:10:41.872 INFO:teuthology.orchestra.run.smithi181.stderr: 35800K .......... .......... .......... .......... .......... 21% 24.2M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 35850K .......... .......... .......... .......... .......... 21% 24.2M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 35900K .......... .......... .......... .......... .......... 21% 48.7M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 35950K .......... .......... .......... .......... .......... 21% 31.5M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36000K .......... .......... .......... .......... .......... 21% 26.2M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36050K .......... .......... .......... .......... .......... 21% 24.0M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36100K .......... .......... .......... .......... .......... 21% 29.7M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36150K .......... .......... .......... .......... .......... 21% 34.9M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36200K .......... .......... .......... .......... .......... 21% 31.3M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36250K .......... .......... .......... .......... .......... 21% 29.2M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36300K .......... .......... .......... .......... .......... 21% 33.5M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36350K .......... .......... .......... .......... .......... 21% 19.5M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36400K .......... .......... .......... .......... .......... 21% 49.3M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36450K .......... .......... .......... .......... .......... 21% 19.6M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36500K .......... .......... .......... .......... .......... 21% 35.9M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36550K .......... .......... .......... .......... .......... 21% 21.3M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36600K .......... .......... .......... .......... .......... 21% 72.1M 10s 2024-03-29T04:10:41.873 INFO:teuthology.orchestra.run.smithi181.stderr: 36650K .......... .......... .......... .......... .......... 21% 26.6M 10s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36700K .......... .......... .......... .......... .......... 21% 27.6M 10s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36750K .......... .......... .......... .......... .......... 21% 23.5M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36800K .......... .......... .......... .......... .......... 21% 24.7M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36850K .......... .......... .......... .......... .......... 21% 35.5M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36900K .......... .......... .......... .......... .......... 21% 33.4M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 36950K .......... .......... .......... .......... .......... 21% 25.3M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37000K .......... .......... .......... .......... .......... 21% 36.2M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37050K .......... .......... .......... .......... .......... 22% 24.0M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37100K .......... .......... .......... .......... .......... 22% 23.5M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37150K .......... .......... .......... .......... .......... 22% 45.9M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37200K .......... .......... .......... .......... .......... 22% 26.4M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37250K .......... .......... .......... .......... .......... 22% 29.9M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37300K .......... .......... .......... .......... .......... 22% 19.9M 9s 2024-03-29T04:10:41.874 INFO:teuthology.orchestra.run.smithi181.stderr: 37350K .......... .......... .......... .......... .......... 22% 33.6M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37400K .......... .......... .......... .......... .......... 22% 48.9M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37450K .......... .......... .......... .......... .......... 22% 25.3M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37500K .......... .......... .......... .......... .......... 22% 30.9M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37550K .......... .......... .......... .......... .......... 22% 40.5M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37600K .......... .......... .......... .......... .......... 22% 18.6M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37650K .......... .......... .......... .......... .......... 22% 41.3M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37700K .......... .......... .......... .......... .......... 22% 18.9M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37750K .......... .......... .......... .......... .......... 22% 29.7M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37800K .......... .......... .......... .......... .......... 22% 22.4M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37850K .......... .......... .......... .......... .......... 22% 50.0M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37900K .......... .......... .......... .......... .......... 22% 50.7M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 37950K .......... .......... .......... .......... .......... 22% 24.9M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 38000K .......... .......... .......... .......... .......... 22% 32.9M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 38050K .......... .......... .......... .......... .......... 22% 25.6M 9s 2024-03-29T04:10:41.875 INFO:teuthology.orchestra.run.smithi181.stderr: 38100K .......... .......... .......... .......... .......... 22% 32.4M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38150K .......... .......... .......... .......... .......... 22% 34.3M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38200K .......... .......... .......... .......... .......... 22% 23.5M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38250K .......... .......... .......... .......... .......... 22% 24.0M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38300K .......... .......... .......... .......... .......... 22% 52.3M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38350K .......... .......... .......... .......... .......... 22% 22.5M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38400K .......... .......... .......... .......... .......... 22% 37.0M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38450K .......... .......... .......... .......... .......... 22% 25.9M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38500K .......... .......... .......... .......... .......... 22% 32.2M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38550K .......... .......... .......... .......... .......... 22% 24.6M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38600K .......... .......... .......... .......... .......... 22% 24.8M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38650K .......... .......... .......... .......... .......... 22% 57.3M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38700K .......... .......... .......... .......... .......... 22% 24.4M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38750K .......... .......... .......... .......... .......... 23% 31.1M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38800K .......... .......... .......... .......... .......... 23% 22.0M 9s 2024-03-29T04:10:41.876 INFO:teuthology.orchestra.run.smithi181.stderr: 38850K .......... .......... .......... .......... .......... 23% 24.3M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 38900K .......... .......... .......... .......... .......... 23% 45.2M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 38950K .......... .......... .......... .......... .......... 23% 32.7M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39000K .......... .......... .......... .......... .......... 23% 21.9M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39050K .......... .......... .......... .......... .......... 23% 31.1M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39100K .......... .......... .......... .......... .......... 23% 24.2M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39150K .......... .......... .......... .......... .......... 23% 10.1M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39200K .......... .......... .......... .......... .......... 23% 55.3M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39250K .......... .......... .......... .......... .......... 23% 83.4M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39300K .......... .......... .......... .......... .......... 23% 32.4M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39350K .......... .......... .......... .......... .......... 23% 30.9M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39400K .......... .......... .......... .......... .......... 23% 23.4M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39450K .......... .......... .......... .......... .......... 23% 22.3M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39500K .......... .......... .......... .......... .......... 23% 27.0M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39550K .......... .......... .......... .......... .......... 23% 17.2M 9s 2024-03-29T04:10:41.877 INFO:teuthology.orchestra.run.smithi181.stderr: 39600K .......... .......... .......... .......... .......... 23% 35.6M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39650K .......... .......... .......... .......... .......... 23% 29.8M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39700K .......... .......... .......... .......... .......... 23% 34.7M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39750K .......... .......... .......... .......... .......... 23% 25.3M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39800K .......... .......... .......... .......... .......... 23% 35.4M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39850K .......... .......... .......... .......... .......... 23% 29.6M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39900K .......... .......... .......... .......... .......... 23% 40.4M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 39950K .......... .......... .......... .......... .......... 23% 23.5M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40000K .......... .......... .......... .......... .......... 23% 28.9M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40050K .......... .......... .......... .......... .......... 23% 28.0M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40100K .......... .......... .......... .......... .......... 23% 64.8M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40150K .......... .......... .......... .......... .......... 23% 34.6M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40200K .......... .......... .......... .......... .......... 23% 23.3M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40250K .......... .......... .......... .......... .......... 23% 23.9M 9s 2024-03-29T04:10:41.878 INFO:teuthology.orchestra.run.smithi181.stderr: 40300K .......... .......... .......... .......... .......... 23% 106M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40350K .......... .......... .......... .......... .......... 23% 110M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40400K .......... .......... .......... .......... .......... 23% 23.5M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40450K .......... .......... .......... .......... .......... 24% 46.2M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40500K .......... .......... .......... .......... .......... 24% 23.8M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40550K .......... .......... .......... .......... .......... 24% 28.4M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40600K .......... .......... .......... .......... .......... 24% 26.2M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40650K .......... .......... .......... .......... .......... 24% 50.8M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40700K .......... .......... .......... .......... .......... 24% 27.4M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40750K .......... .......... .......... .......... .......... 24% 29.5M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40800K .......... .......... .......... .......... .......... 24% 34.7M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40850K .......... .......... .......... .......... .......... 24% 26.8M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40900K .......... .......... .......... .......... .......... 24% 19.9M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 40950K .......... .......... .......... .......... .......... 24% 51.2M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 41000K .......... .......... .......... .......... .......... 24% 23.6M 9s 2024-03-29T04:10:41.879 INFO:teuthology.orchestra.run.smithi181.stderr: 41050K .......... .......... .......... .......... .......... 24% 43.7M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41100K .......... .......... .......... .......... .......... 24% 24.1M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41150K .......... .......... .......... .......... .......... 24% 44.8M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41200K .......... .......... .......... .......... .......... 24% 23.9M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41250K .......... .......... .......... .......... .......... 24% 24.1M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41300K .......... .......... .......... .......... .......... 24% 35.4M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41350K .......... .......... .......... .......... .......... 24% 31.9M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41400K .......... .......... .......... .......... .......... 24% 20.2M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41450K .......... .......... .......... .......... .......... 24% 47.2M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41500K .......... .......... .......... .......... .......... 24% 29.4M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41550K .......... .......... .......... .......... .......... 24% 28.5M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41600K .......... .......... .......... .......... .......... 24% 21.1M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41650K .......... .......... .......... .......... .......... 24% 40.8M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41700K .......... .......... .......... .......... .......... 24% 34.8M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41750K .......... .......... .......... .......... .......... 24% 17.0M 9s 2024-03-29T04:10:41.880 INFO:teuthology.orchestra.run.smithi181.stderr: 41800K .......... .......... .......... .......... .......... 24% 41.6M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 41850K .......... .......... .......... .......... .......... 24% 16.6M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 41900K .......... .......... .......... .......... .......... 24% 24.0M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 41950K .......... .......... .......... .......... .......... 24% 46.3M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42000K .......... .......... .......... .......... .......... 24% 24.4M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42050K .......... .......... .......... .......... .......... 24% 19.8M 9s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42100K .......... .......... .......... .......... .......... 25% 62.2M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42150K .......... .......... .......... .......... .......... 25% 46.4M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42200K .......... .......... .......... .......... .......... 25% 85.5M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42250K .......... .......... .......... .......... .......... 25% 24.2M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42300K .......... .......... .......... .......... .......... 25% 37.1M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42350K .......... .......... .......... .......... .......... 25% 24.5M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42400K .......... .......... .......... .......... .......... 25% 24.3M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42450K .......... .......... .......... .......... .......... 25% 45.7M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42500K .......... .......... .......... .......... .......... 25% 25.2M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42550K .......... .......... .......... .......... .......... 25% 34.2M 8s 2024-03-29T04:10:41.881 INFO:teuthology.orchestra.run.smithi181.stderr: 42600K .......... .......... .......... .......... .......... 25% 24.1M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42650K .......... .......... .......... .......... .......... 25% 26.0M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42700K .......... .......... .......... .......... .......... 25% 53.8M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42750K .......... .......... .......... .......... .......... 25% 23.7M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42800K .......... .......... .......... .......... .......... 25% 32.6M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42850K .......... .......... .......... .......... .......... 25% 23.2M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42900K .......... .......... .......... .......... .......... 25% 34.8M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 42950K .......... .......... .......... .......... .......... 25% 19.6M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43000K .......... .......... .......... .......... .......... 25% 24.1M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43050K .......... .......... .......... .......... .......... 25% 79.6M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43100K .......... .......... .......... .......... .......... 25% 32.1M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43150K .......... .......... .......... .......... .......... 25% 22.4M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43200K .......... .......... .......... .......... .......... 25% 51.2M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43250K .......... .......... .......... .......... .......... 25% 24.3M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43300K .......... .......... .......... .......... .......... 25% 24.3M 8s 2024-03-29T04:10:41.882 INFO:teuthology.orchestra.run.smithi181.stderr: 43350K .......... .......... .......... .......... .......... 25% 26.3M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43400K .......... .......... .......... .......... .......... 25% 37.4M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43450K .......... .......... .......... .......... .......... 25% 36.6M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43500K .......... .......... .......... .......... .......... 25% 27.1M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43550K .......... .......... .......... .......... .......... 25% 32.0M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43600K .......... .......... .......... .......... .......... 25% 23.9M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43650K .......... .......... .......... .......... .......... 25% 29.4M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43700K .......... .......... .......... .......... .......... 25% 32.6M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43750K .......... .......... .......... .......... .......... 25% 19.7M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43800K .......... .......... .......... .......... .......... 26% 26.2M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43850K .......... .......... .......... .......... .......... 26% 62.0M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43900K .......... .......... .......... .......... .......... 26% 40.3M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 43950K .......... .......... .......... .......... .......... 26% 30.5M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 44000K .......... .......... .......... .......... .......... 26% 13.8M 8s 2024-03-29T04:10:41.883 INFO:teuthology.orchestra.run.smithi181.stderr: 44050K .......... .......... .......... .......... .......... 26% 22.9M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44100K .......... .......... .......... .......... .......... 26% 32.2M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44150K .......... .......... .......... .......... .......... 26% 41.9M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44200K .......... .......... .......... .......... .......... 26% 56.2M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44250K .......... .......... .......... .......... .......... 26% 46.3M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44300K .......... .......... .......... .......... .......... 26% 29.7M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44350K .......... .......... .......... .......... .......... 26% 23.3M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44400K .......... .......... .......... .......... .......... 26% 27.1M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44450K .......... .......... .......... .......... .......... 26% 38.9M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44500K .......... .......... .......... .......... .......... 26% 26.3M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44550K .......... .......... .......... .......... .......... 26% 23.7M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44600K .......... .......... .......... .......... .......... 26% 26.7M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44650K .......... .......... .......... .......... .......... 26% 48.8M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44700K .......... .......... .......... .......... .......... 26% 32.7M 8s 2024-03-29T04:10:41.884 INFO:teuthology.orchestra.run.smithi181.stderr: 44750K .......... .......... .......... .......... .......... 26% 23.0M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 44800K .......... .......... .......... .......... .......... 26% 25.1M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 44850K .......... .......... .......... .......... .......... 26% 41.2M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 44900K .......... .......... .......... .......... .......... 26% 25.5M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 44950K .......... .......... .......... .......... .......... 26% 35.5M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45000K .......... .......... .......... .......... .......... 26% 25.2M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45050K .......... .......... .......... .......... .......... 26% 28.6M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45100K .......... .......... .......... .......... .......... 26% 20.0M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45150K .......... .......... .......... .......... .......... 26% 39.5M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45200K .......... .......... .......... .......... .......... 26% 47.2M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45250K .......... .......... .......... .......... .......... 26% 24.7M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45300K .......... .......... .......... .......... .......... 26% 24.1M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45350K .......... .......... .......... .......... .......... 26% 47.8M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45400K .......... .......... .......... .......... .......... 26% 20.7M 8s 2024-03-29T04:10:41.885 INFO:teuthology.orchestra.run.smithi181.stderr: 45450K .......... .......... .......... .......... .......... 26% 60.8M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45500K .......... .......... .......... .......... .......... 27% 22.0M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45550K .......... .......... .......... .......... .......... 27% 26.1M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45600K .......... .......... .......... .......... .......... 27% 24.2M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45650K .......... .......... .......... .......... .......... 27% 31.7M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45700K .......... .......... .......... .......... .......... 27% 25.2M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45750K .......... .......... .......... .......... .......... 27% 52.5M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45800K .......... .......... .......... .......... .......... 27% 29.8M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45850K .......... .......... .......... .......... .......... 27% 23.6M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45900K .......... .......... .......... .......... .......... 27% 52.3M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 45950K .......... .......... .......... .......... .......... 27% 24.7M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 46000K .......... .......... .......... .......... .......... 27% 29.7M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 46050K .......... .......... .......... .......... .......... 27% 24.2M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 46100K .......... .......... .......... .......... .......... 27% 32.0M 8s 2024-03-29T04:10:41.886 INFO:teuthology.orchestra.run.smithi181.stderr: 46150K .......... .......... .......... .......... .......... 27% 24.6M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46200K .......... .......... .......... .......... .......... 27% 29.1M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46250K .......... .......... .......... .......... .......... 27% 35.9M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46300K .......... .......... .......... .......... .......... 27% 25.0M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46350K .......... .......... .......... .......... .......... 27% 37.9M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46400K .......... .......... .......... .......... .......... 27% 23.6M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46450K .......... .......... .......... .......... .......... 27% 24.4M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46500K .......... .......... .......... .......... .......... 27% 50.0M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46550K .......... .......... .......... .......... .......... 27% 34.1M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46600K .......... .......... .......... .......... .......... 27% 24.6M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46650K .......... .......... .......... .......... .......... 27% 24.6M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46700K .......... .......... .......... .......... .......... 27% 23.8M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46750K .......... .......... .......... .......... .......... 27% 49.8M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46800K .......... .......... .......... .......... .......... 27% 32.2M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46850K .......... .......... .......... .......... .......... 27% 21.5M 8s 2024-03-29T04:10:41.887 INFO:teuthology.orchestra.run.smithi181.stderr: 46900K .......... .......... .......... .......... .......... 27% 25.2M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 46950K .......... .......... .......... .......... .......... 27% 44.8M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47000K .......... .......... .......... .......... .......... 27% 32.8M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47050K .......... .......... .......... .......... .......... 27% 24.0M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47100K .......... .......... .......... .......... .......... 27% 22.7M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47150K .......... .......... .......... .......... .......... 28% 40.4M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47200K .......... .......... .......... .......... .......... 28% 22.9M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47250K .......... .......... .......... .......... .......... 28% 53.0M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47300K .......... .......... .......... .......... .......... 28% 22.3M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47350K .......... .......... .......... .......... .......... 28% 26.2M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47400K .......... .......... .......... .......... .......... 28% 48.2M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47450K .......... .......... .......... .......... .......... 28% 24.0M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47500K .......... .......... .......... .......... .......... 28% 33.2M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47550K .......... .......... .......... .......... .......... 28% 24.3M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47600K .......... .......... .......... .......... .......... 28% 40.0M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47650K .......... .......... .......... .......... .......... 28% 24.6M 8s 2024-03-29T04:10:41.888 INFO:teuthology.orchestra.run.smithi181.stderr: 47700K .......... .......... .......... .......... .......... 28% 23.1M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 47750K .......... .......... .......... .......... .......... 28% 48.6M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 47800K .......... .......... .......... .......... .......... 28% 25.4M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 47850K .......... .......... .......... .......... .......... 28% 33.2M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 47900K .......... .......... .......... .......... .......... 28% 23.6M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 47950K .......... .......... .......... .......... .......... 28% 24.7M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48000K .......... .......... .......... .......... .......... 28% 50.8M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48050K .......... .......... .......... .......... .......... 28% 28.2M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48100K .......... .......... .......... .......... .......... 28% 28.8M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48150K .......... .......... .......... .......... .......... 28% 24.3M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48200K .......... .......... .......... .......... .......... 28% 46.2M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48250K .......... .......... .......... .......... .......... 28% 26.8M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48300K .......... .......... .......... .......... .......... 28% 25.7M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48350K .......... .......... .......... .......... .......... 28% 24.0M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48400K .......... .......... .......... .......... .......... 28% 39.7M 8s 2024-03-29T04:10:41.889 INFO:teuthology.orchestra.run.smithi181.stderr: 48450K .......... .......... .......... .......... .......... 28% 35.6M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48500K .......... .......... .......... .......... .......... 28% 25.9M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48550K .......... .......... .......... .......... .......... 28% 28.5M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48600K .......... .......... .......... .......... .......... 28% 28.6M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48650K .......... .......... .......... .......... .......... 28% 26.9M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48700K .......... .......... .......... .......... .......... 28% 27.5M 8s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48750K .......... .......... .......... .......... .......... 28% 34.0M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48800K .......... .......... .......... .......... .......... 28% 25.0M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48850K .......... .......... .......... .......... .......... 29% 23.9M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48900K .......... .......... .......... .......... .......... 29% 43.8M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 48950K .......... .......... .......... .......... .......... 29% 23.6M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 49000K .......... .......... .......... .......... .......... 29% 41.8M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 49050K .......... .......... .......... .......... .......... 29% 25.6M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 49100K .......... .......... .......... .......... .......... 29% 33.3M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 49150K .......... .......... .......... .......... .......... 29% 24.8M 7s 2024-03-29T04:10:41.890 INFO:teuthology.orchestra.run.smithi181.stderr: 49200K .......... .......... .......... .......... .......... 29% 24.5M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49250K .......... .......... .......... .......... .......... 29% 44.2M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49300K .......... .......... .......... .......... .......... 29% 26.4M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49350K .......... .......... .......... .......... .......... 29% 33.7M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49400K .......... .......... .......... .......... .......... 29% 21.3M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49450K .......... .......... .......... .......... .......... 29% 52.2M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49500K .......... .......... .......... .......... .......... 29% 30.8M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49550K .......... .......... .......... .......... .......... 29% 20.5M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49600K .......... .......... .......... .......... .......... 29% 24.8M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49650K .......... .......... .......... .......... .......... 29% 35.0M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49700K .......... .......... .......... .......... .......... 29% 26.3M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49750K .......... .......... .......... .......... .......... 29% 42.5M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49800K .......... .......... .......... .......... .......... 29% 24.8M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49850K .......... .......... .......... .......... .......... 29% 34.6M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49900K .......... .......... .......... .......... .......... 29% 24.8M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 49950K .......... .......... .......... .......... .......... 29% 24.6M 7s 2024-03-29T04:10:41.891 INFO:teuthology.orchestra.run.smithi181.stderr: 50000K .......... .......... .......... .......... .......... 29% 58.5M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50050K .......... .......... .......... .......... .......... 29% 24.5M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50100K .......... .......... .......... .......... .......... 29% 29.4M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50150K .......... .......... .......... .......... .......... 29% 29.0M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50200K .......... .......... .......... .......... .......... 29% 23.4M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50250K .......... .......... .......... .......... .......... 29% 43.0M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50300K .......... .......... .......... .......... .......... 29% 24.3M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50350K .......... .......... .......... .......... .......... 29% 36.0M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50400K .......... .......... .......... .......... .......... 29% 24.5M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50450K .......... .......... .......... .......... .......... 29% 24.4M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50500K .......... .......... .......... .......... .......... 29% 52.9M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50550K .......... .......... .......... .......... .......... 30% 25.2M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50600K .......... .......... .......... .......... .......... 30% 30.3M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50650K .......... .......... .......... .......... .......... 30% 25.5M 7s 2024-03-29T04:10:41.892 INFO:teuthology.orchestra.run.smithi181.stderr: 50700K .......... .......... .......... .......... .......... 30% 24.6M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 50750K .......... .......... .......... .......... .......... 30% 46.2M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 50800K .......... .......... .......... .......... .......... 30% 26.7M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 50850K .......... .......... .......... .......... .......... 30% 23.4M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 50900K .......... .......... .......... .......... .......... 30% 44.3M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 50950K .......... .......... .......... .......... .......... 30% 19.2M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51000K .......... .......... .......... .......... .......... 30% 44.9M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51050K .......... .......... .......... .......... .......... 30% 32.6M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51100K .......... .......... .......... .......... .......... 30% 27.2M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51150K .......... .......... .......... .......... .......... 30% 23.1M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51200K .......... .......... .......... .......... .......... 30% 33.4M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51250K .......... .......... .......... .......... .......... 30% 22.2M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51300K .......... .......... .......... .......... .......... 30% 51.9M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51350K .......... .......... .......... .......... .......... 30% 30.5M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51400K .......... .......... .......... .......... .......... 30% 26.0M 7s 2024-03-29T04:10:41.893 INFO:teuthology.orchestra.run.smithi181.stderr: 51450K .......... .......... .......... .......... .......... 30% 28.8M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51500K .......... .......... .......... .......... .......... 30% 24.0M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51550K .......... .......... .......... .......... .......... 30% 39.2M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51600K .......... .......... .......... .......... .......... 30% 24.8M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51650K .......... .......... .......... .......... .......... 30% 34.4M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51700K .......... .......... .......... .......... .......... 30% 24.9M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51750K .......... .......... .......... .......... .......... 30% 24.3M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51800K .......... .......... .......... .......... .......... 30% 53.9M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51850K .......... .......... .......... .......... .......... 30% 32.6M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51900K .......... .......... .......... .......... .......... 30% 24.5M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 51950K .......... .......... .......... .......... .......... 30% 24.5M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 52000K .......... .......... .......... .......... .......... 30% 50.4M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 52050K .......... .......... .......... .......... .......... 30% 29.2M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 52100K .......... .......... .......... .......... .......... 30% 21.3M 7s 2024-03-29T04:10:41.894 INFO:teuthology.orchestra.run.smithi181.stderr: 52150K .......... .......... .......... .......... .......... 30% 33.6M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52200K .......... .......... .......... .......... .......... 30% 24.6M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52250K .......... .......... .......... .......... .......... 31% 24.8M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52300K .......... .......... .......... .......... .......... 31% 50.1M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52350K .......... .......... .......... .......... .......... 31% 28.8M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52400K .......... .......... .......... .......... .......... 31% 23.2M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52450K .......... .......... .......... .......... .......... 31% 25.7M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52500K .......... .......... .......... .......... .......... 31% 47.8M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52550K .......... .......... .......... .......... .......... 31% 31.1M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52600K .......... .......... .......... .......... .......... 31% 24.8M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52650K .......... .......... .......... .......... .......... 31% 23.5M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52700K .......... .......... .......... .......... .......... 31% 38.8M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52750K .......... .......... .......... .......... .......... 31% 26.4M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52800K .......... .......... .......... .......... .......... 31% 34.9M 7s 2024-03-29T04:10:41.895 INFO:teuthology.orchestra.run.smithi181.stderr: 52850K .......... .......... .......... .......... .......... 31% 25.7M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 52900K .......... .......... .......... .......... .......... 31% 37.6M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 52950K .......... .......... .......... .......... .......... 31% 23.1M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53000K .......... .......... .......... .......... .......... 31% 24.9M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53050K .......... .......... .......... .......... .......... 31% 43.7M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53100K .......... .......... .......... .......... .......... 31% 35.7M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53150K .......... .......... .......... .......... .......... 31% 22.8M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53200K .......... .......... .......... .......... .......... 31% 23.4M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53250K .......... .......... .......... .......... .......... 31% 41.7M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53300K .......... .......... .......... .......... .......... 31% 25.0M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53350K .......... .......... .......... .......... .......... 31% 34.4M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53400K .......... .......... .......... .......... .......... 31% 24.0M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53450K .......... .......... .......... .......... .......... 31% 54.0M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53500K .......... .......... .......... .......... .......... 31% 25.4M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53550K .......... .......... .......... .......... .......... 31% 28.4M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53600K .......... .......... .......... .......... .......... 31% 28.0M 7s 2024-03-29T04:10:41.896 INFO:teuthology.orchestra.run.smithi181.stderr: 53650K .......... .......... .......... .......... .......... 31% 22.3M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53700K .......... .......... .......... .......... .......... 31% 27.0M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53750K .......... .......... .......... .......... .......... 31% 51.1M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53800K .......... .......... .......... .......... .......... 31% 27.3M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53850K .......... .......... .......... .......... .......... 31% 29.8M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53900K .......... .......... .......... .......... .......... 32% 24.6M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 53950K .......... .......... .......... .......... .......... 32% 48.4M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54000K .......... .......... .......... .......... .......... 32% 25.2M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54050K .......... .......... .......... .......... .......... 32% 28.3M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54100K .......... .......... .......... .......... .......... 32% 30.4M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54150K .......... .......... .......... .......... .......... 32% 23.3M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54200K .......... .......... .......... .......... .......... 32% 31.3M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54250K .......... .......... .......... .......... .......... 32% 33.5M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54300K .......... .......... .......... .......... .......... 32% 25.1M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54350K .......... .......... .......... .......... .......... 32% 25.1M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54400K .......... .......... .......... .......... .......... 32% 42.6M 7s 2024-03-29T04:10:41.897 INFO:teuthology.orchestra.run.smithi181.stderr: 54450K .......... .......... .......... .......... .......... 32% 21.1M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54500K .......... .......... .......... .......... .......... 32% 38.4M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54550K .......... .......... .......... .......... .......... 32% 24.9M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54600K .......... .......... .......... .......... .......... 32% 46.3M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54650K .......... .......... .......... .......... .......... 32% 21.7M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54700K .......... .......... .......... .......... .......... 32% 45.5M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54750K .......... .......... .......... .......... .......... 32% 28.8M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54800K .......... .......... .......... .......... .......... 32% 30.6M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54850K .......... .......... .......... .......... .......... 32% 23.4M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54900K .......... .......... .......... .......... .......... 32% 21.7M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 54950K .......... .......... .......... .......... .......... 32% 56.0M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 55000K .......... .......... .......... .......... .......... 32% 32.4M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 55050K .......... .......... .......... .......... .......... 32% 27.6M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 55100K .......... .......... .......... .......... .......... 32% 24.8M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 55150K .......... .......... .......... .......... .......... 32% 24.0M 7s 2024-03-29T04:10:41.898 INFO:teuthology.orchestra.run.smithi181.stderr: 55200K .......... .......... .......... .......... .......... 32% 50.6M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55250K .......... .......... .......... .......... .......... 32% 25.3M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55300K .......... .......... .......... .......... .......... 32% 26.8M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55350K .......... .......... .......... .......... .......... 32% 22.9M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55400K .......... .......... .......... .......... .......... 32% 32.1M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55450K .......... .......... .......... .......... .......... 32% 50.3M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55500K .......... .......... .......... .......... .......... 32% 24.1M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55550K .......... .......... .......... .......... .......... 32% 29.8M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55600K .......... .......... .......... .......... .......... 33% 22.4M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55650K .......... .......... .......... .......... .......... 33% 64.8M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55700K .......... .......... .......... .......... .......... 33% 29.6M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55750K .......... .......... .......... .......... .......... 33% 23.1M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55800K .......... .......... .......... .......... .......... 33% 33.9M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55850K .......... .......... .......... .......... .......... 33% 24.9M 7s 2024-03-29T04:10:41.899 INFO:teuthology.orchestra.run.smithi181.stderr: 55900K .......... .......... .......... .......... .......... 33% 35.9M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 55950K .......... .......... .......... .......... .......... 33% 21.7M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56000K .......... .......... .......... .......... .......... 33% 24.1M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56050K .......... .......... .......... .......... .......... 33% 52.6M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56100K .......... .......... .......... .......... .......... 33% 24.6M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56150K .......... .......... .......... .......... .......... 33% 30.6M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56200K .......... .......... .......... .......... .......... 33% 45.9M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56250K .......... .......... .......... .......... .......... 33% 21.7M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56300K .......... .......... .......... .......... .......... 33% 32.0M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56350K .......... .......... .......... .......... .......... 33% 27.6M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56400K .......... .......... .......... .......... .......... 33% 35.0M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56450K .......... .......... .......... .......... .......... 33% 27.9M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56500K .......... .......... .......... .......... .......... 33% 19.9M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56550K .......... .......... .......... .......... .......... 33% 51.7M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56600K .......... .......... .......... .......... .......... 33% 23.7M 7s 2024-03-29T04:10:41.900 INFO:teuthology.orchestra.run.smithi181.stderr: 56650K .......... .......... .......... .......... .......... 33% 41.5M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56700K .......... .......... .......... .......... .......... 33% 23.0M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56750K .......... .......... .......... .......... .......... 33% 23.5M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56800K .......... .......... .......... .......... .......... 33% 50.5M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56850K .......... .......... .......... .......... .......... 33% 24.8M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56900K .......... .......... .......... .......... .......... 33% 33.8M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 56950K .......... .......... .......... .......... .......... 33% 25.0M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57000K .......... .......... .......... .......... .......... 33% 48.0M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57050K .......... .......... .......... .......... .......... 33% 26.0M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57100K .......... .......... .......... .......... .......... 33% 29.2M 7s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57150K .......... .......... .......... .......... .......... 33% 27.2M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57200K .......... .......... .......... .......... .......... 33% 24.1M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57250K .......... .......... .......... .......... .......... 33% 24.6M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57300K .......... .......... .......... .......... .......... 34% 37.8M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57350K .......... .......... .......... .......... .......... 34% 36.0M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57400K .......... .......... .......... .......... .......... 34% 24.2M 6s 2024-03-29T04:10:41.901 INFO:teuthology.orchestra.run.smithi181.stderr: 57450K .......... .......... .......... .......... .......... 34% 29.9M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57500K .......... .......... .......... .......... .......... 34% 35.4M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57550K .......... .......... .......... .......... .......... 34% 29.5M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57600K .......... .......... .......... .......... .......... 34% 23.1M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57650K .......... .......... .......... .......... .......... 34% 22.7M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57700K .......... .......... .......... .......... .......... 34% 53.4M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57750K .......... .......... .......... .......... .......... 34% 31.9M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57800K .......... .......... .......... .......... .......... 34% 24.4M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57850K .......... .......... .......... .......... .......... 34% 22.5M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57900K .......... .......... .......... .......... .......... 34% 50.7M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 57950K .......... .......... .......... .......... .......... 34% 21.7M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 58000K .......... .......... .......... .......... .......... 34% 40.3M 6s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 58050K .......... .......... .......... .......... .......... 34% 487K 7s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 58100K .......... .......... .......... .......... .......... 34% 426M 7s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 58150K .......... .......... .......... .......... .......... 34% 640M 7s 2024-03-29T04:10:41.902 INFO:teuthology.orchestra.run.smithi181.stderr: 58200K .......... .......... .......... .......... .......... 34% 715M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58250K .......... .......... .......... .......... .......... 34% 683M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58300K .......... .......... .......... .......... .......... 34% 608M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58350K .......... .......... .......... .......... .......... 34% 696M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58400K .......... .......... .......... .......... .......... 34% 704M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58450K .......... .......... .......... .......... .......... 34% 707M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58500K .......... .......... .......... .......... .......... 34% 599M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58550K .......... .......... .......... .......... .......... 34% 717M 7s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58600K .......... .......... .......... .......... .......... 34% 691M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58650K .......... .......... .......... .......... .......... 34% 699M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58700K .......... .......... .......... .......... .......... 34% 620M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58750K .......... .......... .......... .......... .......... 34% 676M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58800K .......... .......... .......... .......... .......... 34% 672M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58850K .......... .......... .......... .......... .......... 34% 676M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58900K .......... .......... .......... .......... .......... 34% 619M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 58950K .......... .......... .......... .......... .......... 35% 707M 6s 2024-03-29T04:10:41.903 INFO:teuthology.orchestra.run.smithi181.stderr: 59000K .......... .......... .......... .......... .......... 35% 472M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59050K .......... .......... .......... .......... .......... 35% 11.9M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59100K .......... .......... .......... .......... .......... 35% 213M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59150K .......... .......... .......... .......... .......... 35% 603M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59200K .......... .......... .......... .......... .......... 35% 691M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59250K .......... .......... .......... .......... .......... 35% 694M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59300K .......... .......... .......... .......... .......... 35% 203M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59350K .......... .......... .......... .......... .......... 35% 202M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59400K .......... .......... .......... .......... .......... 35% 608M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59450K .......... .......... .......... .......... .......... 35% 706M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59500K .......... .......... .......... .......... .......... 35% 720M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59550K .......... .......... .......... .......... .......... 35% 683M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59600K .......... .......... .......... .......... .......... 35% 611M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59650K .......... .......... .......... .......... .......... 35% 711M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59700K .......... .......... .......... .......... .......... 35% 698M 6s 2024-03-29T04:10:41.904 INFO:teuthology.orchestra.run.smithi181.stderr: 59750K .......... .......... .......... .......... .......... 35% 701M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 59800K .......... .......... .......... .......... .......... 35% 129M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 59850K .......... .......... .......... .......... .......... 35% 294M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 59900K .......... .......... .......... .......... .......... 35% 686M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 59950K .......... .......... .......... .......... .......... 35% 605M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60000K .......... .......... .......... .......... .......... 35% 703M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60050K .......... .......... .......... .......... .......... 35% 563M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60100K .......... .......... .......... .......... .......... 35% 743M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60150K .......... .......... .......... .......... .......... 35% 628M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60200K .......... .......... .......... .......... .......... 35% 703M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60250K .......... .......... .......... .......... .......... 35% 554M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60300K .......... .......... .......... .......... .......... 35% 668M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60350K .......... .......... .......... .......... .......... 35% 606M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60400K .......... .......... .......... .......... .......... 35% 600M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60450K .......... .......... .......... .......... .......... 35% 753M 6s 2024-03-29T04:10:41.905 INFO:teuthology.orchestra.run.smithi181.stderr: 60500K .......... .......... .......... .......... .......... 35% 3.43M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60550K .......... .......... .......... .......... .......... 35% 486M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60600K .......... .......... .......... .......... .......... 35% 586M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60650K .......... .......... .......... .......... .......... 36% 670M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60700K .......... .......... .......... .......... .......... 36% 568M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60750K .......... .......... .......... .......... .......... 36% 509M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60800K .......... .......... .......... .......... .......... 36% 581M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60850K .......... .......... .......... .......... .......... 36% 708M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60900K .......... .......... .......... .......... .......... 36% 702M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 60950K .......... .......... .......... .......... .......... 36% 709M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 61000K .......... .......... .......... .......... .......... 36% 632M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 61050K .......... .......... .......... .......... .......... 36% 702M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 61100K .......... .......... .......... .......... .......... 36% 678M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 61150K .......... .......... .......... .......... .......... 36% 682M 6s 2024-03-29T04:10:41.906 INFO:teuthology.orchestra.run.smithi181.stderr: 61200K .......... .......... .......... .......... .......... 36% 607M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61250K .......... .......... .......... .......... .......... 36% 732M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61300K .......... .......... .......... .......... .......... 36% 752M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61350K .......... .......... .......... .......... .......... 36% 3.46M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61400K .......... .......... .......... .......... .......... 36% 36.5M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61450K .......... .......... .......... .......... .......... 36% 27.6M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61500K .......... .......... .......... .......... .......... 36% 110M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61550K .......... .......... .......... .......... .......... 36% 113M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61600K .......... .......... .......... .......... .......... 36% 113M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61650K .......... .......... .......... .......... .......... 36% 114M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61700K .......... .......... .......... .......... .......... 36% 109M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61750K .......... .......... .......... .......... .......... 36% 113M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61800K .......... .......... .......... .......... .......... 36% 106M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61850K .......... .......... .......... .......... .......... 36% 28.1M 6s 2024-03-29T04:10:41.907 INFO:teuthology.orchestra.run.smithi181.stderr: 61900K .......... .......... .......... .......... .......... 36% 22.9M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 61950K .......... .......... .......... .......... .......... 36% 61.3M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62000K .......... .......... .......... .......... .......... 36% 24.4M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62050K .......... .......... .......... .......... .......... 36% 30.2M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62100K .......... .......... .......... .......... .......... 36% 24.8M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62150K .......... .......... .......... .......... .......... 36% 23.9M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62200K .......... .......... .......... .......... .......... 36% 68.7M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62250K .......... .......... .......... .......... .......... 36% 24.4M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62300K .......... .......... .......... .......... .......... 36% 28.6M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62350K .......... .......... .......... .......... .......... 37% 25.1M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62400K .......... .......... .......... .......... .......... 37% 24.8M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62450K .......... .......... .......... .......... .......... 37% 63.7M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62500K .......... .......... .......... .......... .......... 37% 24.6M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62550K .......... .......... .......... .......... .......... 37% 29.2M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62600K .......... .......... .......... .......... .......... 37% 24.9M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62650K .......... .......... .......... .......... .......... 37% 24.4M 6s 2024-03-29T04:10:41.908 INFO:teuthology.orchestra.run.smithi181.stderr: 62700K .......... .......... .......... .......... .......... 37% 68.1M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 62750K .......... .......... .......... .......... .......... 37% 24.3M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 62800K .......... .......... .......... .......... .......... 37% 29.0M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 62850K .......... .......... .......... .......... .......... 37% 19.7M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 62900K .......... .......... .......... .......... .......... 37% 28.1M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 62950K .......... .......... .......... .......... .......... 37% 111M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63000K .......... .......... .......... .......... .......... 37% 24.7M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63050K .......... .......... .......... .......... .......... 37% 20.6M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63100K .......... .......... .......... .......... .......... 37% 24.3M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63150K .......... .......... .......... .......... .......... 37% 24.8M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63200K .......... .......... .......... .......... .......... 37% 112M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63250K .......... .......... .......... .......... .......... 37% 24.5M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63300K .......... .......... .......... .......... .......... 37% 24.5M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63350K .......... .......... .......... .......... .......... 37% 24.9M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63400K .......... .......... .......... .......... .......... 37% 111M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63450K .......... .......... .......... .......... .......... 37% 32.3M 6s 2024-03-29T04:10:41.909 INFO:teuthology.orchestra.run.smithi181.stderr: 63500K .......... .......... .......... .......... .......... 37% 22.0M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63550K .......... .......... .......... .......... .......... 37% 22.1M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63600K .......... .......... .......... .......... .......... 37% 24.6M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63650K .......... .......... .......... .......... .......... 37% 24.5M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63700K .......... .......... .......... .......... .......... 37% 116M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63750K .......... .......... .......... .......... .......... 37% 24.7M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63800K .......... .......... .......... .......... .......... 37% 24.3M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63850K .......... .......... .......... .......... .......... 37% 24.9M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63900K .......... .......... .......... .......... .......... 37% 41.9M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 63950K .......... .......... .......... .......... .......... 37% 27.0M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 64000K .......... .......... .......... .......... .......... 37% 44.3M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 64050K .......... .......... .......... .......... .......... 38% 24.6M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 64100K .......... .......... .......... .......... .......... 38% 33.5M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 64150K .......... .......... .......... .......... .......... 38% 24.4M 6s 2024-03-29T04:10:41.910 INFO:teuthology.orchestra.run.smithi181.stderr: 64200K .......... .......... .......... .......... .......... 38% 24.8M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64250K .......... .......... .......... .......... .......... 38% 52.6M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64300K .......... .......... .......... .......... .......... 38% 29.1M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64350K .......... .......... .......... .......... .......... 38% 26.2M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64400K .......... .......... .......... .......... .......... 38% 25.7M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64450K .......... .......... .......... .......... .......... 38% 23.3M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64500K .......... .......... .......... .......... .......... 38% 51.1M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64550K .......... .......... .......... .......... .......... 38% 25.6M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64600K .......... .......... .......... .......... .......... 38% 32.2M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64650K .......... .......... .......... .......... .......... 38% 24.6M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64700K .......... .......... .......... .......... .......... 38% 24.9M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64750K .......... .......... .......... .......... .......... 38% 53.4M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64800K .......... .......... .......... .......... .......... 38% 33.3M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64850K .......... .......... .......... .......... .......... 38% 24.1M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64900K .......... .......... .......... .......... .......... 38% 25.2M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 64950K .......... .......... .......... .......... .......... 38% 22.3M 6s 2024-03-29T04:10:41.911 INFO:teuthology.orchestra.run.smithi181.stderr: 65000K .......... .......... .......... .......... .......... 38% 57.9M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65050K .......... .......... .......... .......... .......... 38% 30.6M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65100K .......... .......... .......... .......... .......... 38% 25.5M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65150K .......... .......... .......... .......... .......... 38% 24.8M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65200K .......... .......... .......... .......... .......... 38% 24.9M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65250K .......... .......... .......... .......... .......... 38% 53.8M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65300K .......... .......... .......... .......... .......... 38% 32.9M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65350K .......... .......... .......... .......... .......... 38% 25.7M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65400K .......... .......... .......... .......... .......... 38% 24.5M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65450K .......... .......... .......... .......... .......... 38% 25.3M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65500K .......... .......... .......... .......... .......... 38% 43.1M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65550K .......... .......... .......... .......... .......... 38% 30.0M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65600K .......... .......... .......... .......... .......... 38% 24.3M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65650K .......... .......... .......... .......... .......... 38% 26.5M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65700K .......... .......... .......... .......... .......... 39% 25.0M 6s 2024-03-29T04:10:41.912 INFO:teuthology.orchestra.run.smithi181.stderr: 65750K .......... .......... .......... .......... .......... 39% 57.2M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 65800K .......... .......... .......... .......... .......... 39% 30.1M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 65850K .......... .......... .......... .......... .......... 39% 26.3M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 65900K .......... .......... .......... .......... .......... 39% 25.5M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 65950K .......... .......... .......... .......... .......... 39% 24.5M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66000K .......... .......... .......... .......... .......... 39% 48.2M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66050K .......... .......... .......... .......... .......... 39% 29.7M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66100K .......... .......... .......... .......... .......... 39% 27.7M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66150K .......... .......... .......... .......... .......... 39% 24.8M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66200K .......... .......... .......... .......... .......... 39% 24.2M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66250K .......... .......... .......... .......... .......... 39% 55.0M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66300K .......... .......... .......... .......... .......... 39% 28.4M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66350K .......... .......... .......... .......... .......... 39% 27.9M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66400K .......... .......... .......... .......... .......... 39% 24.4M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66450K .......... .......... .......... .......... .......... 39% 24.0M 6s 2024-03-29T04:10:41.913 INFO:teuthology.orchestra.run.smithi181.stderr: 66500K .......... .......... .......... .......... .......... 39% 60.1M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66550K .......... .......... .......... .......... .......... 39% 24.0M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66600K .......... .......... .......... .......... .......... 39% 29.4M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66650K .......... .......... .......... .......... .......... 39% 24.7M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66700K .......... .......... .......... .......... .......... 39% 24.2M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66750K .......... .......... .......... .......... .......... 39% 59.4M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66800K .......... .......... .......... .......... .......... 39% 26.7M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66850K .......... .......... .......... .......... .......... 39% 26.2M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66900K .......... .......... .......... .......... .......... 39% 26.9M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 66950K .......... .......... .......... .......... .......... 39% 24.9M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 67000K .......... .......... .......... .......... .......... 39% 54.6M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 67050K .......... .......... .......... .......... .......... 39% 25.8M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 67100K .......... .......... .......... .......... .......... 39% 30.0M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 67150K .......... .......... .......... .......... .......... 39% 24.8M 6s 2024-03-29T04:10:41.914 INFO:teuthology.orchestra.run.smithi181.stderr: 67200K .......... .......... .......... .......... .......... 39% 24.0M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67250K .......... .......... .......... .......... .......... 39% 57.0M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67300K .......... .......... .......... .......... .......... 39% 24.3M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67350K .......... .......... .......... .......... .......... 39% 31.4M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67400K .......... .......... .......... .......... .......... 40% 24.4M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67450K .......... .......... .......... .......... .......... 40% 24.5M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67500K .......... .......... .......... .......... .......... 40% 57.8M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67550K .......... .......... .......... .......... .......... 40% 22.8M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67600K .......... .......... .......... .......... .......... 40% 34.7M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67650K .......... .......... .......... .......... .......... 40% 21.2M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67700K .......... .......... .......... .......... .......... 40% 27.8M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67750K .......... .......... .......... .......... .......... 40% 59.0M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67800K .......... .......... .......... .......... .......... 40% 23.1M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67850K .......... .......... .......... .......... .......... 40% 36.6M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67900K .......... .......... .......... .......... .......... 40% 23.8M 6s 2024-03-29T04:10:41.915 INFO:teuthology.orchestra.run.smithi181.stderr: 67950K .......... .......... .......... .......... .......... 40% 25.5M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68000K .......... .......... .......... .......... .......... 40% 48.1M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68050K .......... .......... .......... .......... .......... 40% 23.6M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68100K .......... .......... .......... .......... .......... 40% 24.9M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68150K .......... .......... .......... .......... .......... 40% 36.1M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68200K .......... .......... .......... .......... .......... 40% 24.7M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68250K .......... .......... .......... .......... .......... 40% 47.5M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68300K .......... .......... .......... .......... .......... 40% 24.9M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68350K .......... .......... .......... .......... .......... 40% 9.85M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68400K .......... .......... .......... .......... .......... 40% 286M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68450K .......... .......... .......... .......... .......... 40% 199M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68500K .......... .......... .......... .......... .......... 40% 47.5M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68550K .......... .......... .......... .......... .......... 40% 25.6M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68600K .......... .......... .......... .......... .......... 40% 32.2M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68650K .......... .......... .......... .......... .......... 40% 22.1M 5s 2024-03-29T04:10:41.916 INFO:teuthology.orchestra.run.smithi181.stderr: 68700K .......... .......... .......... .......... .......... 40% 24.8M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 68750K .......... .......... .......... .......... .......... 40% 52.7M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 68800K .......... .......... .......... .......... .......... 40% 24.5M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 68850K .......... .......... .......... .......... .......... 40% 31.7M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 68900K .......... .......... .......... .......... .......... 40% 24.4M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 68950K .......... .......... .......... .......... .......... 40% 23.8M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69000K .......... .......... .......... .......... .......... 40% 64.1M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69050K .......... .......... .......... .......... .......... 40% 25.0M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69100K .......... .......... .......... .......... .......... 41% 28.6M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69150K .......... .......... .......... .......... .......... 41% 24.4M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69200K .......... .......... .......... .......... .......... 41% 24.7M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69250K .......... .......... .......... .......... .......... 41% 54.8M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69300K .......... .......... .......... .......... .......... 41% 26.9M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69350K .......... .......... .......... .......... .......... 41% 28.7M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69400K .......... .......... .......... .......... .......... 41% 25.0M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69450K .......... .......... .......... .......... .......... 41% 23.5M 5s 2024-03-29T04:10:41.917 INFO:teuthology.orchestra.run.smithi181.stderr: 69500K .......... .......... .......... .......... .......... 41% 54.0M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69550K .......... .......... .......... .......... .......... 41% 27.9M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69600K .......... .......... .......... .......... .......... 41% 30.1M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69650K .......... .......... .......... .......... .......... 41% 23.3M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69700K .......... .......... .......... .......... .......... 41% 24.7M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69750K .......... .......... .......... .......... .......... 41% 35.8M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69800K .......... .......... .......... .......... .......... 41% 36.8M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69850K .......... .......... .......... .......... .......... 41% 26.1M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69900K .......... .......... .......... .......... .......... 41% 25.7M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 69950K .......... .......... .......... .......... .......... 41% 49.4M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70000K .......... .......... .......... .......... .......... 41% 25.1M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70050K .......... .......... .......... .......... .......... 41% 29.7M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70100K .......... .......... .......... .......... .......... 41% 27.6M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70150K .......... .......... .......... .......... .......... 41% 23.6M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70200K .......... .......... .......... .......... .......... 41% 24.1M 5s 2024-03-29T04:10:41.918 INFO:teuthology.orchestra.run.smithi181.stderr: 70250K .......... .......... .......... .......... .......... 41% 54.3M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70300K .......... .......... .......... .......... .......... 41% 30.6M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70350K .......... .......... .......... .......... .......... 41% 24.9M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70400K .......... .......... .......... .......... .......... 41% 24.3M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70450K .......... .......... .......... .......... .......... 41% 47.4M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70500K .......... .......... .......... .......... .......... 41% 20.1M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70550K .......... .......... .......... .......... .......... 41% 45.5M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70600K .......... .......... .......... .......... .......... 41% 24.4M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70650K .......... .......... .......... .......... .......... 41% 25.1M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70700K .......... .......... .......... .......... .......... 41% 53.8M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70750K .......... .......... .......... .......... .......... 42% 22.3M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70800K .......... .......... .......... .......... .......... 42% 35.8M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70850K .......... .......... .......... .......... .......... 42% 25.9M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70900K .......... .......... .......... .......... .......... 42% 24.5M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 70950K .......... .......... .......... .......... .......... 42% 42.8M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 71000K .......... .......... .......... .......... .......... 42% 24.7M 5s 2024-03-29T04:10:41.919 INFO:teuthology.orchestra.run.smithi181.stderr: 71050K .......... .......... .......... .......... .......... 42% 35.0M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71100K .......... .......... .......... .......... .......... 42% 25.0M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71150K .......... .......... .......... .......... .......... 42% 39.2M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71200K .......... .......... .......... .......... .......... 42% 25.8M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71250K .......... .......... .......... .......... .......... 42% 23.6M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71300K .......... .......... .......... .......... .......... 42% 42.4M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71350K .......... .......... .......... .......... .......... 42% 24.2M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71400K .......... .......... .......... .......... .......... 42% 37.6M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71450K .......... .......... .......... .......... .......... 42% 26.0M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71500K .......... .......... .......... .......... .......... 42% 22.0M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71550K .......... .......... .......... .......... .......... 42% 49.7M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71600K .......... .......... .......... .......... .......... 42% 24.2M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71650K .......... .......... .......... .......... .......... 42% 37.3M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71700K .......... .......... .......... .......... .......... 42% 25.2M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71750K .......... .......... .......... .......... .......... 42% 24.2M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71800K .......... .......... .......... .......... .......... 42% 50.9M 5s 2024-03-29T04:10:41.920 INFO:teuthology.orchestra.run.smithi181.stderr: 71850K .......... .......... .......... .......... .......... 42% 23.8M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 71900K .......... .......... .......... .......... .......... 42% 28.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 71950K .......... .......... .......... .......... .......... 42% 28.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72000K .......... .......... .......... .......... .......... 42% 24.7M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72050K .......... .......... .......... .......... .......... 42% 50.4M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72100K .......... .......... .......... .......... .......... 42% 25.2M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72150K .......... .......... .......... .......... .......... 42% 33.0M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72200K .......... .......... .......... .......... .......... 42% 24.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72250K .......... .......... .......... .......... .......... 42% 24.4M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72300K .......... .......... .......... .......... .......... 42% 50.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72350K .......... .......... .......... .......... .......... 42% 24.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72400K .......... .......... .......... .......... .......... 42% 32.7M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72450K .......... .......... .......... .......... .......... 43% 24.9M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72500K .......... .......... .......... .......... .......... 43% 24.4M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72550K .......... .......... .......... .......... .......... 43% 53.8M 5s 2024-03-29T04:10:41.921 INFO:teuthology.orchestra.run.smithi181.stderr: 72600K .......... .......... .......... .......... .......... 43% 27.7M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72650K .......... .......... .......... .......... .......... 43% 20.9M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72700K .......... .......... .......... .......... .......... 43% 37.3M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72750K .......... .......... .......... .......... .......... 43% 24.3M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72800K .......... .......... .......... .......... .......... 43% 48.6M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72850K .......... .......... .......... .......... .......... 43% 27.5M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72900K .......... .......... .......... .......... .......... 43% 28.9M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 72950K .......... .......... .......... .......... .......... 43% 24.9M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73000K .......... .......... .......... .......... .......... 43% 25.1M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73050K .......... .......... .......... .......... .......... 43% 46.8M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73100K .......... .......... .......... .......... .......... 43% 24.2M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73150K .......... .......... .......... .......... .......... 43% 35.2M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73200K .......... .......... .......... .......... .......... 43% 24.6M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73250K .......... .......... .......... .......... .......... 43% 37.9M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73300K .......... .......... .......... .......... .......... 43% 30.8M 5s 2024-03-29T04:10:41.922 INFO:teuthology.orchestra.run.smithi181.stderr: 73350K .......... .......... .......... .......... .......... 43% 24.2M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73400K .......... .......... .......... .......... .......... 43% 23.5M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73450K .......... .......... .......... .......... .......... 43% 35.7M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73500K .......... .......... .......... .......... .......... 43% 24.7M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73550K .......... .......... .......... .......... .......... 43% 48.8M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73600K .......... .......... .......... .......... .......... 43% 24.4M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73650K .......... .......... .......... .......... .......... 43% 33.8M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73700K .......... .......... .......... .......... .......... 43% 24.7M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73750K .......... .......... .......... .......... .......... 43% 24.6M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73800K .......... .......... .......... .......... .......... 43% 50.5M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73850K .......... .......... .......... .......... .......... 43% 23.6M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73900K .......... .......... .......... .......... .......... 43% 36.6M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 73950K .......... .......... .......... .......... .......... 43% 24.5M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 74000K .......... .......... .......... .......... .......... 43% 24.5M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 74050K .......... .......... .......... .......... .......... 43% 47.5M 5s 2024-03-29T04:10:41.923 INFO:teuthology.orchestra.run.smithi181.stderr: 74100K .......... .......... .......... .......... .......... 43% 25.5M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74150K .......... .......... .......... .......... .......... 44% 33.3M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74200K .......... .......... .......... .......... .......... 44% 24.7M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74250K .......... .......... .......... .......... .......... 44% 25.0M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74300K .......... .......... .......... .......... .......... 44% 23.3M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74350K .......... .......... .......... .......... .......... 44% 45.7M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74400K .......... .......... .......... .......... .......... 44% 32.6M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74450K .......... .......... .......... .......... .......... 44% 25.3M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74500K .......... .......... .......... .......... .......... 44% 24.6M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74550K .......... .......... .......... .......... .......... 44% 24.5M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74600K .......... .......... .......... .......... .......... 44% 53.3M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74650K .......... .......... .......... .......... .......... 44% 28.9M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74700K .......... .......... .......... .......... .......... 44% 27.1M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74750K .......... .......... .......... .......... .......... 44% 24.7M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74800K .......... .......... .......... .......... .......... 44% 24.9M 5s 2024-03-29T04:10:41.924 INFO:teuthology.orchestra.run.smithi181.stderr: 74850K .......... .......... .......... .......... .......... 44% 51.2M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 74900K .......... .......... .......... .......... .......... 44% 28.4M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 74950K .......... .......... .......... .......... .......... 44% 28.1M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75000K .......... .......... .......... .......... .......... 44% 24.5M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75050K .......... .......... .......... .......... .......... 44% 24.4M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75100K .......... .......... .......... .......... .......... 44% 52.5M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75150K .......... .......... .......... .......... .......... 44% 29.7M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75200K .......... .......... .......... .......... .......... 44% 27.0M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75250K .......... .......... .......... .......... .......... 44% 25.0M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75300K .......... .......... .......... .......... .......... 44% 24.6M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75350K .......... .......... .......... .......... .......... 44% 52.3M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75400K .......... .......... .......... .......... .......... 44% 23.3M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75450K .......... .......... .......... .......... .......... 44% 32.6M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75500K .......... .......... .......... .......... .......... 44% 25.9M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75550K .......... .......... .......... .......... .......... 44% 24.1M 5s 2024-03-29T04:10:41.925 INFO:teuthology.orchestra.run.smithi181.stderr: 75600K .......... .......... .......... .......... .......... 44% 58.4M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75650K .......... .......... .......... .......... .......... 44% 25.8M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75700K .......... .......... .......... .......... .......... 44% 30.1M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75750K .......... .......... .......... .......... .......... 44% 24.4M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75800K .......... .......... .......... .......... .......... 44% 24.8M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75850K .......... .......... .......... .......... .......... 45% 55.3M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75900K .......... .......... .......... .......... .......... 45% 25.5M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 75950K .......... .......... .......... .......... .......... 45% 29.9M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76000K .......... .......... .......... .......... .......... 45% 24.5M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76050K .......... .......... .......... .......... .......... 45% 25.1M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76100K .......... .......... .......... .......... .......... 45% 60.0M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76150K .......... .......... .......... .......... .......... 45% 22.2M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76200K .......... .......... .......... .......... .......... 45% 32.5M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76250K .......... .......... .......... .......... .......... 45% 23.3M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76300K .......... .......... .......... .......... .......... 45% 24.2M 5s 2024-03-29T04:10:41.926 INFO:teuthology.orchestra.run.smithi181.stderr: 76350K .......... .......... .......... .......... .......... 45% 54.1M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76400K .......... .......... .......... .......... .......... 45% 25.2M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76450K .......... .......... .......... .......... .......... 45% 32.0M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76500K .......... .......... .......... .......... .......... 45% 24.8M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76550K .......... .......... .......... .......... .......... 45% 24.4M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76600K .......... .......... .......... .......... .......... 45% 49.5M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76650K .......... .......... .......... .......... .......... 45% 24.1M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76700K .......... .......... .......... .......... .......... 45% 35.3M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76750K .......... .......... .......... .......... .......... 45% 24.8M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76800K .......... .......... .......... .......... .......... 45% 24.5M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76850K .......... .......... .......... .......... .......... 45% 50.3M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76900K .......... .......... .......... .......... .......... 45% 24.0M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 76950K .......... .......... .......... .......... .......... 45% 35.2M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 77000K .......... .......... .......... .......... .......... 45% 24.7M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 77050K .......... .......... .......... .......... .......... 45% 24.5M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 77100K .......... .......... .......... .......... .......... 45% 47.8M 5s 2024-03-29T04:10:41.927 INFO:teuthology.orchestra.run.smithi181.stderr: 77150K .......... .......... .......... .......... .......... 45% 24.6M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77200K .......... .......... .......... .......... .......... 45% 36.1M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77250K .......... .......... .......... .......... .......... 45% 24.2M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77300K .......... .......... .......... .......... .......... 45% 23.5M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77350K .......... .......... .......... .......... .......... 45% 49.1M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77400K .......... .......... .......... .......... .......... 45% 25.5M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77450K .......... .......... .......... .......... .......... 45% 32.2M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77500K .......... .......... .......... .......... .......... 46% 24.2M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77550K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77600K .......... .......... .......... .......... .......... 46% 54.1M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77650K .......... .......... .......... .......... .......... 46% 25.3M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77700K .......... .......... .......... .......... .......... 46% 31.9M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77750K .......... .......... .......... .......... .......... 46% 24.1M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77800K .......... .......... .......... .......... .......... 46% 24.2M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77850K .......... .......... .......... .......... .......... 46% 58.7M 5s 2024-03-29T04:10:41.928 INFO:teuthology.orchestra.run.smithi181.stderr: 77900K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 77950K .......... .......... .......... .......... .......... 46% 30.9M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78000K .......... .......... .......... .......... .......... 46% 24.5M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78050K .......... .......... .......... .......... .......... 46% 23.5M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78100K .......... .......... .......... .......... .......... 46% 61.4M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78150K .......... .......... .......... .......... .......... 46% 24.0M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78200K .......... .......... .......... .......... .......... 46% 29.5M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78250K .......... .......... .......... .......... .......... 46% 21.2M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78300K .......... .......... .......... .......... .......... 46% 50.7M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78350K .......... .......... .......... .......... .......... 46% 28.6M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78400K .......... .......... .......... .......... .......... 46% 24.4M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78450K .......... .......... .......... .......... .......... 46% 28.9M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78500K .......... .......... .......... .......... .......... 46% 24.2M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78550K .......... .......... .......... .......... .......... 46% 48.9M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78600K .......... .......... .......... .......... .......... 46% 29.0M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78650K .......... .......... .......... .......... .......... 46% 24.8M 5s 2024-03-29T04:10:41.929 INFO:teuthology.orchestra.run.smithi181.stderr: 78700K .......... .......... .......... .......... .......... 46% 27.5M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 78750K .......... .......... .......... .......... .......... 46% 25.0M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 78800K .......... .......... .......... .......... .......... 46% 51.0M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 78850K .......... .......... .......... .......... .......... 46% 28.8M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 78900K .......... .......... .......... .......... .......... 46% 26.4M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 78950K .......... .......... .......... .......... .......... 46% 26.0M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79000K .......... .......... .......... .......... .......... 46% 24.0M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79050K .......... .......... .......... .......... .......... 46% 45.7M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79100K .......... .......... .......... .......... .......... 46% 32.6M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79150K .......... .......... .......... .......... .......... 46% 25.2M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79200K .......... .......... .......... .......... .......... 47% 24.8M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79250K .......... .......... .......... .......... .......... 47% 24.9M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79300K .......... .......... .......... .......... .......... 47% 39.9M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79350K .......... .......... .......... .......... .......... 47% 34.2M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79400K .......... .......... .......... .......... .......... 47% 24.1M 5s 2024-03-29T04:10:41.930 INFO:teuthology.orchestra.run.smithi181.stderr: 79450K .......... .......... .......... .......... .......... 47% 27.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79500K .......... .......... .......... .......... .......... 47% 23.9M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79550K .......... .......... .......... .......... .......... 47% 48.5M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79600K .......... .......... .......... .......... .......... 47% 24.1M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79650K .......... .......... .......... .......... .......... 47% 36.1M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79700K .......... .......... .......... .......... .......... 47% 24.4M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79750K .......... .......... .......... .......... .......... 47% 23.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79800K .......... .......... .......... .......... .......... 47% 39.8M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79850K .......... .......... .......... .......... .......... 47% 26.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79900K .......... .......... .......... .......... .......... 47% 36.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 79950K .......... .......... .......... .......... .......... 47% 24.1M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 80000K .......... .......... .......... .......... .......... 47% 24.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 80050K .......... .......... .......... .......... .......... 47% 45.0M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 80100K .......... .......... .......... .......... .......... 47% 23.1M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 80150K .......... .......... .......... .......... .......... 47% 39.7M 5s 2024-03-29T04:10:41.931 INFO:teuthology.orchestra.run.smithi181.stderr: 80200K .......... .......... .......... .......... .......... 47% 24.4M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80250K .......... .......... .......... .......... .......... 47% 40.6M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80300K .......... .......... .......... .......... .......... 47% 23.3M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80350K .......... .......... .......... .......... .......... 47% 25.0M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80400K .......... .......... .......... .......... .......... 47% 15.7M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80450K .......... .......... .......... .......... .......... 47% 105M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80500K .......... .......... .......... .......... .......... 47% 56.8M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80550K .......... .......... .......... .......... .......... 47% 24.5M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80600K .......... .......... .......... .......... .......... 47% 24.8M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80650K .......... .......... .......... .......... .......... 47% 47.3M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80700K .......... .......... .......... .......... .......... 47% 32.4M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80750K .......... .......... .......... .......... .......... 47% 26.2M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80800K .......... .......... .......... .......... .......... 47% 25.1M 5s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80850K .......... .......... .......... .......... .......... 47% 23.6M 4s 2024-03-29T04:10:41.932 INFO:teuthology.orchestra.run.smithi181.stderr: 80900K .......... .......... .......... .......... .......... 48% 52.9M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 80950K .......... .......... .......... .......... .......... 48% 32.9M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81000K .......... .......... .......... .......... .......... 48% 24.6M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81050K .......... .......... .......... .......... .......... 48% 25.5M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81100K .......... .......... .......... .......... .......... 48% 24.1M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81150K .......... .......... .......... .......... .......... 48% 48.5M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81200K .......... .......... .......... .......... .......... 48% 24.8M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81250K .......... .......... .......... .......... .......... 48% 34.1M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81300K .......... .......... .......... .......... .......... 48% 24.1M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81350K .......... .......... .......... .......... .......... 48% 24.8M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81400K .......... .......... .......... .......... .......... 48% 53.2M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81450K .......... .......... .......... .......... .......... 48% 29.9M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81500K .......... .......... .......... .......... .......... 48% 28.7M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81550K .......... .......... .......... .......... .......... 48% 23.5M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81600K .......... .......... .......... .......... .......... 48% 24.6M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81650K .......... .......... .......... .......... .......... 48% 51.8M 4s 2024-03-29T04:10:41.933 INFO:teuthology.orchestra.run.smithi181.stderr: 81700K .......... .......... .......... .......... .......... 48% 26.7M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 81750K .......... .......... .......... .......... .......... 48% 27.4M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 81800K .......... .......... .......... .......... .......... 48% 24.1M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 81850K .......... .......... .......... .......... .......... 48% 24.9M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 81900K .......... .......... .......... .......... .......... 48% 58.5M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 81950K .......... .......... .......... .......... .......... 48% 24.9M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82000K .......... .......... .......... .......... .......... 48% 31.2M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82050K .......... .......... .......... .......... .......... 48% 31.9M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82100K .......... .......... .......... .......... .......... 48% 19.8M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82150K .......... .......... .......... .......... .......... 48% 49.5M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82200K .......... .......... .......... .......... .......... 48% 24.3M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82250K .......... .......... .......... .......... .......... 48% 35.4M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82300K .......... .......... .......... .......... .......... 48% 25.0M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82350K .......... .......... .......... .......... .......... 48% 29.8M 4s 2024-03-29T04:10:41.934 INFO:teuthology.orchestra.run.smithi181.stderr: 82400K .......... .......... .......... .......... .......... 48% 34.2M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82450K .......... .......... .......... .......... .......... 48% 24.4M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82500K .......... .......... .......... .......... .......... 48% 36.3M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82550K .......... .......... .......... .......... .......... 49% 25.2M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82600K .......... .......... .......... .......... .......... 49% 37.9M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82650K .......... .......... .......... .......... .......... 49% 29.3M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82700K .......... .......... .......... .......... .......... 49% 23.3M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82750K .......... .......... .......... .......... .......... 49% 35.3M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82800K .......... .......... .......... .......... .......... 49% 24.8M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82850K .......... .......... .......... .......... .......... 49% 24.5M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82900K .......... .......... .......... .......... .......... 49% 52.7M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 82950K .......... .......... .......... .......... .......... 49% 14.3M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 83000K .......... .......... .......... .......... .......... 49% 110M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 83050K .......... .......... .......... .......... .......... 49% 30.4M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 83100K .......... .......... .......... .......... .......... 49% 24.9M 4s 2024-03-29T04:10:41.935 INFO:teuthology.orchestra.run.smithi181.stderr: 83150K .......... .......... .......... .......... .......... 49% 50.3M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83200K .......... .......... .......... .......... .......... 49% 24.8M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83250K .......... .......... .......... .......... .......... 49% 31.0M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83300K .......... .......... .......... .......... .......... 49% 25.8M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83350K .......... .......... .......... .......... .......... 49% 25.0M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83400K .......... .......... .......... .......... .......... 49% 50.8M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83450K .......... .......... .......... .......... .......... 49% 24.6M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83500K .......... .......... .......... .......... .......... 49% 32.9M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83550K .......... .......... .......... .......... .......... 49% 24.7M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83600K .......... .......... .......... .......... .......... 49% 24.6M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83650K .......... .......... .......... .......... .......... 49% 53.1M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83700K .......... .......... .......... .......... .......... 49% 24.7M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83750K .......... .......... .......... .......... .......... 49% 33.3M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83800K .......... .......... .......... .......... .......... 49% 24.4M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83850K .......... .......... .......... .......... .......... 49% 24.6M 4s 2024-03-29T04:10:41.936 INFO:teuthology.orchestra.run.smithi181.stderr: 83900K .......... .......... .......... .......... .......... 49% 53.2M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 83950K .......... .......... .......... .......... .......... 49% 24.1M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84000K .......... .......... .......... .......... .......... 49% 30.2M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84050K .......... .......... .......... .......... .......... 49% 26.6M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84100K .......... .......... .......... .......... .......... 49% 24.6M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84150K .......... .......... .......... .......... .......... 49% 54.8M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84200K .......... .......... .......... .......... .......... 49% 27.9M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84250K .......... .......... .......... .......... .......... 50% 20.9M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84300K .......... .......... .......... .......... .......... 50% 30.1M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84350K .......... .......... .......... .......... .......... 50% 27.4M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84400K .......... .......... .......... .......... .......... 50% 56.7M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84450K .......... .......... .......... .......... .......... 50% 27.5M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84500K .......... .......... .......... .......... .......... 50% 23.9M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84550K .......... .......... .......... .......... .......... 50% 29.8M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84600K .......... .......... .......... .......... .......... 50% 34.9M 4s 2024-03-29T04:10:41.937 INFO:teuthology.orchestra.run.smithi181.stderr: 84650K .......... .......... .......... .......... .......... 50% 31.8M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84700K .......... .......... .......... .......... .......... 50% 25.1M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84750K .......... .......... .......... .......... .......... 50% 27.0M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84800K .......... .......... .......... .......... .......... 50% 28.6M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84850K .......... .......... .......... .......... .......... 50% 22.6M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84900K .......... .......... .......... .......... .......... 50% 26.4M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 84950K .......... .......... .......... .......... .......... 50% 68.1M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85000K .......... .......... .......... .......... .......... 50% 23.0M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85050K .......... .......... .......... .......... .......... 50% 32.0M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85100K .......... .......... .......... .......... .......... 50% 24.9M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85150K .......... .......... .......... .......... .......... 50% 24.6M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85200K .......... .......... .......... .......... .......... 50% 46.4M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85250K .......... .......... .......... .......... .......... 50% 26.9M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85300K .......... .......... .......... .......... .......... 50% 31.5M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85350K .......... .......... .......... .......... .......... 50% 24.3M 4s 2024-03-29T04:10:41.938 INFO:teuthology.orchestra.run.smithi181.stderr: 85400K .......... .......... .......... .......... .......... 50% 24.3M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85450K .......... .......... .......... .......... .......... 50% 51.5M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85500K .......... .......... .......... .......... .......... 50% 25.5M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85550K .......... .......... .......... .......... .......... 50% 32.2M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85600K .......... .......... .......... .......... .......... 50% 24.2M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85650K .......... .......... .......... .......... .......... 50% 25.0M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85700K .......... .......... .......... .......... .......... 50% 53.0M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85750K .......... .......... .......... .......... .......... 50% 23.0M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85800K .......... .......... .......... .......... .......... 50% 32.8M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85850K .......... .......... .......... .......... .......... 50% 22.4M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85900K .......... .......... .......... .......... .......... 50% 24.3M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 85950K .......... .......... .......... .......... .......... 51% 47.0M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 86000K .......... .......... .......... .......... .......... 51% 25.4M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 86050K .......... .......... .......... .......... .......... 51% 34.6M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 86100K .......... .......... .......... .......... .......... 51% 24.1M 4s 2024-03-29T04:10:41.939 INFO:teuthology.orchestra.run.smithi181.stderr: 86150K .......... .......... .......... .......... .......... 51% 24.9M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86200K .......... .......... .......... .......... .......... 51% 45.9M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86250K .......... .......... .......... .......... .......... 51% 24.0M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86300K .......... .......... .......... .......... .......... 51% 37.0M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86350K .......... .......... .......... .......... .......... 51% 24.6M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86400K .......... .......... .......... .......... .......... 51% 24.9M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86450K .......... .......... .......... .......... .......... 51% 46.4M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86500K .......... .......... .......... .......... .......... 51% 25.2M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86550K .......... .......... .......... .......... .......... 51% 28.0M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86600K .......... .......... .......... .......... .......... 51% 27.6M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86650K .......... .......... .......... .......... .......... 51% 25.2M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86700K .......... .......... .......... .......... .......... 51% 54.1M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86750K .......... .......... .......... .......... .......... 51% 29.9M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86800K .......... .......... .......... .......... .......... 51% 24.3M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86850K .......... .......... .......... .......... .......... 51% 33.0M 4s 2024-03-29T04:10:41.940 INFO:teuthology.orchestra.run.smithi181.stderr: 86900K .......... .......... .......... .......... .......... 51% 24.0M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 86950K .......... .......... .......... .......... .......... 51% 35.1M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87000K .......... .......... .......... .......... .......... 51% 26.4M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87050K .......... .......... .......... .......... .......... 51% 30.9M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87100K .......... .......... .......... .......... .......... 51% 19.7M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87150K .......... .......... .......... .......... .......... 51% 103M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87200K .......... .......... .......... .......... .......... 51% 26.6M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87250K .......... .......... .......... .......... .......... 51% 29.0M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87300K .......... .......... .......... .......... .......... 51% 24.3M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87350K .......... .......... .......... .......... .......... 51% 24.5M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87400K .......... .......... .......... .......... .......... 51% 58.1M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87450K .......... .......... .......... .......... .......... 51% 26.6M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87500K .......... .......... .......... .......... .......... 51% 26.2M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87550K .......... .......... .......... .......... .......... 51% 26.7M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87600K .......... .......... .......... .......... .......... 51% 24.4M 4s 2024-03-29T04:10:41.941 INFO:teuthology.orchestra.run.smithi181.stderr: 87650K .......... .......... .......... .......... .......... 52% 54.5M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87700K .......... .......... .......... .......... .......... 52% 27.2M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87750K .......... .......... .......... .......... .......... 52% 28.2M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87800K .......... .......... .......... .......... .......... 52% 24.0M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87850K .......... .......... .......... .......... .......... 52% 26.1M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87900K .......... .......... .......... .......... .......... 52% 36.7M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 87950K .......... .......... .......... .......... .......... 52% 35.8M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88000K .......... .......... .......... .......... .......... 52% 24.9M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88050K .......... .......... .......... .......... .......... 52% 24.7M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88100K .......... .......... .......... .......... .......... 52% 24.7M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88150K .......... .......... .......... .......... .......... 52% 48.6M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88200K .......... .......... .......... .......... .......... 52% 31.4M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88250K .......... .......... .......... .......... .......... 52% 26.0M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88300K .......... .......... .......... .......... .......... 52% 23.7M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88350K .......... .......... .......... .......... .......... 52% 24.5M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88400K .......... .......... .......... .......... .......... 52% 49.7M 4s 2024-03-29T04:10:41.942 INFO:teuthology.orchestra.run.smithi181.stderr: 88450K .......... .......... .......... .......... .......... 52% 32.0M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88500K .......... .......... .......... .......... .......... 52% 24.2M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88550K .......... .......... .......... .......... .......... 52% 25.7M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88600K .......... .......... .......... .......... .......... 52% 25.9M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88650K .......... .......... .......... .......... .......... 52% 48.4M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88700K .......... .......... .......... .......... .......... 52% 31.2M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88750K .......... .......... .......... .......... .......... 52% 24.0M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88800K .......... .......... .......... .......... .......... 52% 23.6M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88850K .......... .......... .......... .......... .......... 52% 50.5M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88900K .......... .......... .......... .......... .......... 52% 24.4M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 88950K .......... .......... .......... .......... .......... 52% 36.3M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 89000K .......... .......... .......... .......... .......... 52% 23.7M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 89050K .......... .......... .......... .......... .......... 52% 24.5M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 89100K .......... .......... .......... .......... .......... 52% 35.6M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 89150K .......... .......... .......... .......... .......... 52% 25.2M 4s 2024-03-29T04:10:41.943 INFO:teuthology.orchestra.run.smithi181.stderr: 89200K .......... .......... .......... .......... .......... 52% 49.8M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89250K .......... .......... .......... .......... .......... 52% 23.6M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89300K .......... .......... .......... .......... .......... 53% 24.2M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89350K .......... .......... .......... .......... .......... 53% 35.6M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89400K .......... .......... .......... .......... .......... 53% 25.0M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89450K .......... .......... .......... .......... .......... 53% 46.7M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89500K .......... .......... .......... .......... .......... 53% 24.3M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89550K .......... .......... .......... .......... .......... 53% 23.5M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89600K .......... .......... .......... .......... .......... 53% 36.7M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89650K .......... .......... .......... .......... .......... 53% 24.8M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89700K .......... .......... .......... .......... .......... 53% 45.4M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89750K .......... .......... .......... .......... .......... 53% 24.3M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89800K .......... .......... .......... .......... .......... 53% 24.1M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89850K .......... .......... .......... .......... .......... 53% 34.8M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89900K .......... .......... .......... .......... .......... 53% 24.9M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 89950K .......... .......... .......... .......... .......... 53% 46.4M 4s 2024-03-29T04:10:41.944 INFO:teuthology.orchestra.run.smithi181.stderr: 90000K .......... .......... .......... .......... .......... 53% 25.1M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90050K .......... .......... .......... .......... .......... 53% 34.0M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90100K .......... .......... .......... .......... .......... 53% 24.8M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90150K .......... .......... .......... .......... .......... 53% 28.9M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90200K .......... .......... .......... .......... .......... 53% 21.3M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90250K .......... .......... .......... .......... .......... 53% 50.4M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90300K .......... .......... .......... .......... .......... 53% 33.7M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90350K .......... .......... .......... .......... .......... 53% 24.9M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90400K .......... .......... .......... .......... .......... 53% 24.2M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90450K .......... .......... .......... .......... .......... 53% 24.7M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90500K .......... .......... .......... .......... .......... 53% 51.0M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90550K .......... .......... .......... .......... .......... 53% 31.4M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90600K .......... .......... .......... .......... .......... 53% 25.7M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90650K .......... .......... .......... .......... .......... 53% 24.3M 4s 2024-03-29T04:10:41.945 INFO:teuthology.orchestra.run.smithi181.stderr: 90700K .......... .......... .......... .......... .......... 53% 25.1M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 90750K .......... .......... .......... .......... .......... 53% 52.2M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 90800K .......... .......... .......... .......... .......... 53% 28.6M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 90850K .......... .......... .......... .......... .......... 53% 27.5M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 90900K .......... .......... .......... .......... .......... 53% 24.5M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 90950K .......... .......... .......... .......... .......... 53% 25.3M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91000K .......... .......... .......... .......... .......... 54% 57.2M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91050K .......... .......... .......... .......... .......... 54% 21.9M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91100K .......... .......... .......... .......... .......... 54% 35.2M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91150K .......... .......... .......... .......... .......... 54% 25.0M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91200K .......... .......... .......... .......... .......... 54% 23.3M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91250K .......... .......... .......... .......... .......... 54% 49.3M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91300K .......... .......... .......... .......... .......... 54% 24.6M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91350K .......... .......... .......... .......... .......... 54% 37.9M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91400K .......... .......... .......... .......... .......... 54% 23.1M 4s 2024-03-29T04:10:41.946 INFO:teuthology.orchestra.run.smithi181.stderr: 91450K .......... .......... .......... .......... .......... 54% 25.0M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91500K .......... .......... .......... .......... .......... 54% 49.6M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91550K .......... .......... .......... .......... .......... 54% 23.5M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91600K .......... .......... .......... .......... .......... 54% 41.2M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91650K .......... .......... .......... .......... .......... 54% 22.3M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91700K .......... .......... .......... .......... .......... 54% 25.0M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91750K .......... .......... .......... .......... .......... 54% 48.9M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91800K .......... .......... .......... .......... .......... 54% 34.4M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91850K .......... .......... .......... .......... .......... 54% 23.1M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91900K .......... .......... .......... .......... .......... 54% 26.6M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 91950K .......... .......... .......... .......... .......... 54% 24.5M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 92000K .......... .......... .......... .......... .......... 54% 46.6M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 92050K .......... .......... .......... .......... .......... 54% 24.7M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 92100K .......... .......... .......... .......... .......... 54% 35.3M 4s 2024-03-29T04:10:41.947 INFO:teuthology.orchestra.run.smithi181.stderr: 92150K .......... .......... .......... .......... .......... 54% 25.0M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92200K .......... .......... .......... .......... .......... 54% 23.9M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92250K .......... .......... .......... .......... .......... 54% 50.4M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92300K .......... .......... .......... .......... .......... 54% 34.0M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92350K .......... .......... .......... .......... .......... 54% 24.3M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92400K .......... .......... .......... .......... .......... 54% 24.8M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92450K .......... .......... .......... .......... .......... 54% 30.5M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92500K .......... .......... .......... .......... .......... 54% 34.7M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92550K .......... .......... .......... .......... .......... 54% 24.9M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92600K .......... .......... .......... .......... .......... 54% 33.9M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92650K .......... .......... .......... .......... .......... 54% 24.5M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92700K .......... .......... .......... .......... .......... 55% 24.6M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92750K .......... .......... .......... .......... .......... 55% 51.8M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92800K .......... .......... .......... .......... .......... 55% 24.4M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92850K .......... .......... .......... .......... .......... 55% 33.6M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92900K .......... .......... .......... .......... .......... 55% 24.1M 4s 2024-03-29T04:10:41.948 INFO:teuthology.orchestra.run.smithi181.stderr: 92950K .......... .......... .......... .......... .......... 55% 25.2M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93000K .......... .......... .......... .......... .......... 55% 50.1M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93050K .......... .......... .......... .......... .......... 55% 31.9M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93100K .......... .......... .......... .......... .......... 55% 25.8M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93150K .......... .......... .......... .......... .......... 55% 24.7M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93200K .......... .......... .......... .......... .......... 55% 24.6M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93250K .......... .......... .......... .......... .......... 55% 51.3M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93300K .......... .......... .......... .......... .......... 55% 31.2M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93350K .......... .......... .......... .......... .......... 55% 25.8M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93400K .......... .......... .......... .......... .......... 55% 24.6M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93450K .......... .......... .......... .......... .......... 55% 24.6M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93500K .......... .......... .......... .......... .......... 55% 52.9M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93550K .......... .......... .......... .......... .......... 55% 27.6M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93600K .......... .......... .......... .......... .......... 55% 28.9M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93650K .......... .......... .......... .......... .......... 55% 24.6M 4s 2024-03-29T04:10:41.949 INFO:teuthology.orchestra.run.smithi181.stderr: 93700K .......... .......... .......... .......... .......... 55% 24.7M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 93750K .......... .......... .......... .......... .......... 55% 54.0M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 93800K .......... .......... .......... .......... .......... 55% 27.8M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 93850K .......... .......... .......... .......... .......... 55% 27.9M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 93900K .......... .......... .......... .......... .......... 55% 27.0M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 93950K .......... .......... .......... .......... .......... 55% 22.6M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94000K .......... .......... .......... .......... .......... 55% 54.3M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94050K .......... .......... .......... .......... .......... 55% 27.8M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94100K .......... .......... .......... .......... .......... 55% 20.9M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94150K .......... .......... .......... .......... .......... 55% 37.7M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94200K .......... .......... .......... .......... .......... 55% 24.1M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94250K .......... .......... .......... .......... .......... 55% 53.3M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94300K .......... .......... .......... .......... .......... 55% 25.6M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94350K .......... .......... .......... .......... .......... 56% 30.2M 4s 2024-03-29T04:10:41.950 INFO:teuthology.orchestra.run.smithi181.stderr: 94400K .......... .......... .......... .......... .......... 56% 25.2M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94450K .......... .......... .......... .......... .......... 56% 24.3M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94500K .......... .......... .......... .......... .......... 56% 55.5M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94550K .......... .......... .......... .......... .......... 56% 24.4M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94600K .......... .......... .......... .......... .......... 56% 24.6M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94650K .......... .......... .......... .......... .......... 56% 32.2M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94700K .......... .......... .......... .......... .......... 56% 27.9M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94750K .......... .......... .......... .......... .......... 56% 36.9M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94800K .......... .......... .......... .......... .......... 56% 27.2M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94850K .......... .......... .......... .......... .......... 56% 25.2M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94900K .......... .......... .......... .......... .......... 56% 31.1M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 94950K .......... .......... .......... .......... .......... 56% 24.5M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 95000K .......... .......... .......... .......... .......... 56% 53.9M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 95050K .......... .......... .......... .......... .......... 56% 24.9M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 95100K .......... .......... .......... .......... .......... 56% 24.3M 4s 2024-03-29T04:10:41.951 INFO:teuthology.orchestra.run.smithi181.stderr: 95150K .......... .......... .......... .......... .......... 56% 32.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95200K .......... .......... .......... .......... .......... 56% 24.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95250K .......... .......... .......... .......... .......... 56% 51.1M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95300K .......... .......... .......... .......... .......... 56% 23.7M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95350K .......... .......... .......... .......... .......... 56% 26.0M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95400K .......... .......... .......... .......... .......... 56% 32.2M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95450K .......... .......... .......... .......... .......... 56% 24.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95500K .......... .......... .......... .......... .......... 56% 24.8M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95550K .......... .......... .......... .......... .......... 56% 46.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95600K .......... .......... .......... .......... .......... 56% 24.6M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95650K .......... .......... .......... .......... .......... 56% 34.0M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95700K .......... .......... .......... .......... .......... 56% 24.7M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95750K .......... .......... .......... .......... .......... 56% 24.3M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95800K .......... .......... .......... .......... .......... 56% 48.8M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95850K .......... .......... .......... .......... .......... 56% 24.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95900K .......... .......... .......... .......... .......... 56% 35.5M 4s 2024-03-29T04:10:41.952 INFO:teuthology.orchestra.run.smithi181.stderr: 95950K .......... .......... .......... .......... .......... 56% 24.4M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96000K .......... .......... .......... .......... .......... 56% 24.7M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96050K .......... .......... .......... .......... .......... 57% 48.0M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96100K .......... .......... .......... .......... .......... 57% 24.6M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96150K .......... .......... .......... .......... .......... 57% 32.9M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96200K .......... .......... .......... .......... .......... 57% 25.2M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96250K .......... .......... .......... .......... .......... 57% 49.8M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96300K .......... .......... .......... .......... .......... 57% 25.2M 4s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96350K .......... .......... .......... .......... .......... 57% 24.5M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96400K .......... .......... .......... .......... .......... 57% 32.2M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96450K .......... .......... .......... .......... .......... 57% 22.9M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96500K .......... .......... .......... .......... .......... 57% 67.1M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96550K .......... .......... .......... .......... .......... 57% 24.3M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96600K .......... .......... .......... .......... .......... 57% 30.8M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96650K .......... .......... .......... .......... .......... 57% 24.9M 3s 2024-03-29T04:10:41.953 INFO:teuthology.orchestra.run.smithi181.stderr: 96700K .......... .......... .......... .......... .......... 57% 24.8M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 96750K .......... .......... .......... .......... .......... 57% 54.2M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 96800K .......... .......... .......... .......... .......... 57% 25.0M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 96850K .......... .......... .......... .......... .......... 57% 30.5M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 96900K .......... .......... .......... .......... .......... 57% 24.3M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 96950K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97000K .......... .......... .......... .......... .......... 57% 30.6M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97050K .......... .......... .......... .......... .......... 57% 40.8M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97100K .......... .......... .......... .......... .......... 57% 28.7M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97150K .......... .......... .......... .......... .......... 57% 24.9M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97200K .......... .......... .......... .......... .......... 57% 25.1M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97250K .......... .......... .......... .......... .......... 57% 52.7M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97300K .......... .......... .......... .......... .......... 57% 23.9M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97350K .......... .......... .......... .......... .......... 57% 33.5M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97400K .......... .......... .......... .......... .......... 57% 24.0M 3s 2024-03-29T04:10:41.954 INFO:teuthology.orchestra.run.smithi181.stderr: 97450K .......... .......... .......... .......... .......... 57% 24.6M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97500K .......... .......... .......... .......... .......... 57% 55.2M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97550K .......... .......... .......... .......... .......... 57% 27.3M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97600K .......... .......... .......... .......... .......... 57% 27.6M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97650K .......... .......... .......... .......... .......... 57% 24.8M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97700K .......... .......... .......... .......... .......... 57% 23.7M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97750K .......... .......... .......... .......... .......... 58% 49.4M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97800K .......... .......... .......... .......... .......... 58% 28.5M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97850K .......... .......... .......... .......... .......... 58% 26.9M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97900K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 97950K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 98000K .......... .......... .......... .......... .......... 58% 57.3M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 98050K .......... .......... .......... .......... .......... 58% 30.7M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 98100K .......... .......... .......... .......... .......... 58% 10.5M 3s 2024-03-29T04:10:41.955 INFO:teuthology.orchestra.run.smithi181.stderr: 98150K .......... .......... .......... .......... .......... 58% 127M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98200K .......... .......... .......... .......... .......... 58% 54.6M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98250K .......... .......... .......... .......... .......... 58% 57.0M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98300K .......... .......... .......... .......... .......... 58% 31.2M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98350K .......... .......... .......... .......... .......... 58% 25.6M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98400K .......... .......... .......... .......... .......... 58% 24.0M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98450K .......... .......... .......... .......... .......... 58% 45.2M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98500K .......... .......... .......... .......... .......... 58% 25.2M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98550K .......... .......... .......... .......... .......... 58% 33.6M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98600K .......... .......... .......... .......... .......... 58% 23.4M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98650K .......... .......... .......... .......... .......... 58% 26.2M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98700K .......... .......... .......... .......... .......... 58% 42.3M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98750K .......... .......... .......... .......... .......... 58% 23.5M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98800K .......... .......... .......... .......... .......... 58% 36.4M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98850K .......... .......... .......... .......... .......... 58% 25.6M 3s 2024-03-29T04:10:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: 98900K .......... .......... .......... .......... .......... 58% 24.3M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 98950K .......... .......... .......... .......... .......... 58% 42.9M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99000K .......... .......... .......... .......... .......... 58% 25.5M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99050K .......... .......... .......... .......... .......... 58% 34.6M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99100K .......... .......... .......... .......... .......... 58% 24.9M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99150K .......... .......... .......... .......... .......... 58% 25.0M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99200K .......... .......... .......... .......... .......... 58% 43.9M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99250K .......... .......... .......... .......... .......... 58% 24.6M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99300K .......... .......... .......... .......... .......... 58% 36.6M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99350K .......... .......... .......... .......... .......... 58% 25.1M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99400K .......... .......... .......... .......... .......... 58% 36.3M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99450K .......... .......... .......... .......... .......... 59% 25.2M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99500K .......... .......... .......... .......... .......... 59% 25.7M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99550K .......... .......... .......... .......... .......... 59% 39.9M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99600K .......... .......... .......... .......... .......... 59% 25.3M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99650K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-03-29T04:10:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: 99700K .......... .......... .......... .......... .......... 59% 35.5M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr: 99750K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr: 99800K .......... .......... .......... .......... .......... 59% 47.1M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr: 99850K .......... .......... .......... .......... .......... 59% 24.4M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr: 99900K .......... .......... .......... .......... .......... 59% 35.5M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr: 99950K .......... .......... .......... .......... .......... 59% 24.4M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100000K .......... .......... .......... .......... .......... 59% 24.9M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100050K .......... .......... .......... .......... .......... 59% 48.4M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100100K .......... .......... .......... .......... .......... 59% 25.1M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100150K .......... .......... .......... .......... .......... 59% 33.3M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100200K .......... .......... .......... .......... .......... 59% 23.9M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100250K .......... .......... .......... .......... .......... 59% 23.7M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100300K .......... .......... .......... .......... .......... 59% 55.0M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100350K .......... .......... .......... .......... .......... 59% 24.3M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100400K .......... .......... .......... .......... .......... 59% 26.3M 3s 2024-03-29T04:10:41.958 INFO:teuthology.orchestra.run.smithi181.stderr:100450K .......... .......... .......... .......... .......... 59% 31.5M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100500K .......... .......... .......... .......... .......... 59% 24.0M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100550K .......... .......... .......... .......... .......... 59% 23.9M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100600K .......... .......... .......... .......... .......... 59% 52.9M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100650K .......... .......... .......... .......... .......... 59% 29.6M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100700K .......... .......... .......... .......... .......... 59% 25.5M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100750K .......... .......... .......... .......... .......... 59% 24.5M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100800K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100850K .......... .......... .......... .......... .......... 59% 61.0M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100900K .......... .......... .......... .......... .......... 59% 23.6M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:100950K .......... .......... .......... .......... .......... 59% 31.0M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101000K .......... .......... .......... .......... .......... 59% 25.8M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101050K .......... .......... .......... .......... .......... 59% 24.4M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101100K .......... .......... .......... .......... .......... 60% 46.3M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101150K .......... .......... .......... .......... .......... 60% 25.6M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101200K .......... .......... .......... .......... .......... 60% 34.2M 3s 2024-03-29T04:10:41.959 INFO:teuthology.orchestra.run.smithi181.stderr:101250K .......... .......... .......... .......... .......... 60% 24.5M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101300K .......... .......... .......... .......... .......... 60% 24.7M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101350K .......... .......... .......... .......... .......... 60% 47.8M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101400K .......... .......... .......... .......... .......... 60% 23.2M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101450K .......... .......... .......... .......... .......... 60% 35.3M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101500K .......... .......... .......... .......... .......... 60% 24.8M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101550K .......... .......... .......... .......... .......... 60% 24.7M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101600K .......... .......... .......... .......... .......... 60% 46.9M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101650K .......... .......... .......... .......... .......... 60% 24.3M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101700K .......... .......... .......... .......... .......... 60% 35.5M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101750K .......... .......... .......... .......... .......... 60% 24.7M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101800K .......... .......... .......... .......... .......... 60% 25.2M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101850K .......... .......... .......... .......... .......... 60% 56.5M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101900K .......... .......... .......... .......... .......... 60% 31.2M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:101950K .......... .......... .......... .......... .......... 60% 24.2M 3s 2024-03-29T04:10:41.960 INFO:teuthology.orchestra.run.smithi181.stderr:102000K .......... .......... .......... .......... .......... 60% 24.9M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102050K .......... .......... .......... .......... .......... 60% 24.5M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102100K .......... .......... .......... .......... .......... 60% 53.0M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102150K .......... .......... .......... .......... .......... 60% 23.8M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102200K .......... .......... .......... .......... .......... 60% 33.8M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102250K .......... .......... .......... .......... .......... 60% 24.9M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102300K .......... .......... .......... .......... .......... 60% 23.1M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102350K .......... .......... .......... .......... .......... 60% 51.1M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102400K .......... .......... .......... .......... .......... 60% 33.7M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102450K .......... .......... .......... .......... .......... 60% 24.6M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102500K .......... .......... .......... .......... .......... 60% 24.6M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102550K .......... .......... .......... .......... .......... 60% 24.5M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102600K .......... .......... .......... .......... .......... 60% 55.7M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102650K .......... .......... .......... .......... .......... 60% 23.8M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102700K .......... .......... .......... .......... .......... 60% 33.9M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102750K .......... .......... .......... .......... .......... 60% 24.5M 3s 2024-03-29T04:10:41.961 INFO:teuthology.orchestra.run.smithi181.stderr:102800K .......... .......... .......... .......... .......... 61% 25.2M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:102850K .......... .......... .......... .......... .......... 61% 49.4M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:102900K .......... .......... .......... .......... .......... 61% 33.0M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:102950K .......... .......... .......... .......... .......... 61% 24.8M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103000K .......... .......... .......... .......... .......... 61% 23.2M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103050K .......... .......... .......... .......... .......... 61% 24.7M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103100K .......... .......... .......... .......... .......... 61% 49.9M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103150K .......... .......... .......... .......... .......... 61% 31.4M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103200K .......... .......... .......... .......... .......... 61% 25.8M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103250K .......... .......... .......... .......... .......... 61% 25.3M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103300K .......... .......... .......... .......... .......... 61% 24.1M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103350K .......... .......... .......... .......... .......... 61% 51.7M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103400K .......... .......... .......... .......... .......... 61% 25.4M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103450K .......... .......... .......... .......... .......... 61% 31.9M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103500K .......... .......... .......... .......... .......... 61% 24.8M 3s 2024-03-29T04:10:41.962 INFO:teuthology.orchestra.run.smithi181.stderr:103550K .......... .......... .......... .......... .......... 61% 24.3M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103600K .......... .......... .......... .......... .......... 61% 54.8M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103650K .......... .......... .......... .......... .......... 61% 28.9M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103700K .......... .......... .......... .......... .......... 61% 26.9M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103750K .......... .......... .......... .......... .......... 61% 26.1M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103800K .......... .......... .......... .......... .......... 61% 23.6M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103850K .......... .......... .......... .......... .......... 61% 54.9M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103900K .......... .......... .......... .......... .......... 61% 28.8M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:103950K .......... .......... .......... .......... .......... 61% 26.7M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104000K .......... .......... .......... .......... .......... 61% 25.0M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104050K .......... .......... .......... .......... .......... 61% 25.0M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104100K .......... .......... .......... .......... .......... 61% 51.7M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104150K .......... .......... .......... .......... .......... 61% 28.1M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104200K .......... .......... .......... .......... .......... 61% 28.3M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104250K .......... .......... .......... .......... .......... 61% 24.5M 3s 2024-03-29T04:10:41.963 INFO:teuthology.orchestra.run.smithi181.stderr:104300K .......... .......... .......... .......... .......... 61% 22.9M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104350K .......... .......... .......... .......... .......... 61% 63.9M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104400K .......... .......... .......... .......... .......... 61% 26.6M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104450K .......... .......... .......... .......... .......... 61% 21.5M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104500K .......... .......... .......... .......... .......... 62% 33.6M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104550K .......... .......... .......... .......... .......... 62% 26.0M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104600K .......... .......... .......... .......... .......... 62% 55.9M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104650K .......... .......... .......... .......... .......... 62% 25.1M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104700K .......... .......... .......... .......... .......... 62% 24.8M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104750K .......... .......... .......... .......... .......... 62% 30.8M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104800K .......... .......... .......... .......... .......... 62% 24.5M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104850K .......... .......... .......... .......... .......... 62% 54.2M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104900K .......... .......... .......... .......... .......... 62% 25.0M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:104950K .......... .......... .......... .......... .......... 62% 23.3M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:105000K .......... .......... .......... .......... .......... 62% 33.2M 3s 2024-03-29T04:10:41.964 INFO:teuthology.orchestra.run.smithi181.stderr:105050K .......... .......... .......... .......... .......... 62% 24.5M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105100K .......... .......... .......... .......... .......... 62% 53.6M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105150K .......... .......... .......... .......... .......... 62% 24.7M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105200K .......... .......... .......... .......... .......... 62% 24.8M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105250K .......... .......... .......... .......... .......... 62% 30.7M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105300K .......... .......... .......... .......... .......... 62% 25.2M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105350K .......... .......... .......... .......... .......... 62% 47.9M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105400K .......... .......... .......... .......... .......... 62% 25.0M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105450K .......... .......... .......... .......... .......... 62% 25.1M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105500K .......... .......... .......... .......... .......... 62% 33.2M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105550K .......... .......... .......... .......... .......... 62% 25.0M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105600K .......... .......... .......... .......... .......... 62% 48.4M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105650K .......... .......... .......... .......... .......... 62% 24.6M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105700K .......... .......... .......... .......... .......... 62% 22.9M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105750K .......... .......... .......... .......... .......... 62% 37.2M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105800K .......... .......... .......... .......... .......... 62% 22.7M 3s 2024-03-29T04:10:41.965 INFO:teuthology.orchestra.run.smithi181.stderr:105850K .......... .......... .......... .......... .......... 62% 24.4M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:105900K .......... .......... .......... .......... .......... 62% 46.4M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:105950K .......... .......... .......... .......... .......... 62% 24.1M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106000K .......... .......... .......... .......... .......... 62% 37.5M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106050K .......... .......... .......... .......... .......... 62% 24.3M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106100K .......... .......... .......... .......... .......... 62% 23.7M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106150K .......... .......... .......... .......... .......... 63% 48.4M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106200K .......... .......... .......... .......... .......... 63% 24.2M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106250K .......... .......... .......... .......... .......... 63% 36.7M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106300K .......... .......... .......... .......... .......... 63% 15.2M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106350K .......... .......... .......... .......... .......... 63% 65.9M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106400K .......... .......... .......... .......... .......... 63% 46.3M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106450K .......... .......... .......... .......... .......... 63% 33.0M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106500K .......... .......... .......... .......... .......... 63% 24.8M 3s 2024-03-29T04:10:41.966 INFO:teuthology.orchestra.run.smithi181.stderr:106550K .......... .......... .......... .......... .......... 63% 24.4M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106600K .......... .......... .......... .......... .......... 63% 52.0M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106650K .......... .......... .......... .......... .......... 63% 19.7M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106700K .......... .......... .......... .......... .......... 63% 42.1M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106750K .......... .......... .......... .......... .......... 63% 25.5M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106800K .......... .......... .......... .......... .......... 63% 24.9M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106850K .......... .......... .......... .......... .......... 63% 25.0M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106900K .......... .......... .......... .......... .......... 63% 49.3M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:106950K .......... .......... .......... .......... .......... 63% 23.7M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107000K .......... .......... .......... .......... .......... 63% 34.0M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107050K .......... .......... .......... .......... .......... 63% 24.4M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107100K .......... .......... .......... .......... .......... 63% 55.2M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107150K .......... .......... .......... .......... .......... 63% 24.4M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107200K .......... .......... .......... .......... .......... 63% 29.6M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107250K .......... .......... .......... .......... .......... 63% 25.7M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107300K .......... .......... .......... .......... .......... 63% 25.5M 3s 2024-03-29T04:10:41.967 INFO:teuthology.orchestra.run.smithi181.stderr:107350K .......... .......... .......... .......... .......... 63% 53.3M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107400K .......... .......... .......... .......... .......... 63% 25.1M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107450K .......... .......... .......... .......... .......... 63% 31.0M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107500K .......... .......... .......... .......... .......... 63% 24.2M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107550K .......... .......... .......... .......... .......... 63% 23.8M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107600K .......... .......... .......... .......... .......... 63% 56.7M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107650K .......... .......... .......... .......... .......... 63% 27.2M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107700K .......... .......... .......... .......... .......... 63% 27.7M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107750K .......... .......... .......... .......... .......... 63% 26.3M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107800K .......... .......... .......... .......... .......... 63% 24.4M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107850K .......... .......... .......... .......... .......... 64% 50.9M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107900K .......... .......... .......... .......... .......... 64% 27.9M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:107950K .......... .......... .......... .......... .......... 64% 25.1M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:108000K .......... .......... .......... .......... .......... 64% 26.0M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:108050K .......... .......... .......... .......... .......... 64% 23.6M 3s 2024-03-29T04:10:41.968 INFO:teuthology.orchestra.run.smithi181.stderr:108100K .......... .......... .......... .......... .......... 64% 36.3M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108150K .......... .......... .......... .......... .......... 64% 44.1M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108200K .......... .......... .......... .......... .......... 64% 26.4M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108250K .......... .......... .......... .......... .......... 64% 23.8M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108300K .......... .......... .......... .......... .......... 64% 25.9M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108350K .......... .......... .......... .......... .......... 64% 47.9M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108400K .......... .......... .......... .......... .......... 64% 30.8M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108450K .......... .......... .......... .......... .......... 64% 26.1M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108500K .......... .......... .......... .......... .......... 64% 24.3M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108550K .......... .......... .......... .......... .......... 64% 23.7M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108600K .......... .......... .......... .......... .......... 64% 50.3M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108650K .......... .......... .......... .......... .......... 64% 34.0M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108700K .......... .......... .......... .......... .......... 64% 22.7M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108750K .......... .......... .......... .......... .......... 64% 37.0M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108800K .......... .......... .......... .......... .......... 64% 30.8M 3s 2024-03-29T04:10:41.969 INFO:teuthology.orchestra.run.smithi181.stderr:108850K .......... .......... .......... .......... .......... 64% 23.8M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:108900K .......... .......... .......... .......... .......... 64% 35.1M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:108950K .......... .......... .......... .......... .......... 64% 24.4M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109000K .......... .......... .......... .......... .......... 64% 20.6M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109050K .......... .......... .......... .......... .......... 64% 32.8M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109100K .......... .......... .......... .......... .......... 64% 37.9M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109150K .......... .......... .......... .......... .......... 64% 33.7M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109200K .......... .......... .......... .......... .......... 64% 24.9M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109250K .......... .......... .......... .......... .......... 64% 26.3M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109300K .......... .......... .......... .......... .......... 64% 32.1M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109350K .......... .......... .......... .......... .......... 64% 24.2M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109400K .......... .......... .......... .......... .......... 64% 50.7M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109450K .......... .......... .......... .......... .......... 64% 24.2M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109500K .......... .......... .......... .......... .......... 64% 24.6M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109550K .......... .......... .......... .......... .......... 65% 39.9M 3s 2024-03-29T04:10:41.970 INFO:teuthology.orchestra.run.smithi181.stderr:109600K .......... .......... .......... .......... .......... 65% 22.7M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109650K .......... .......... .......... .......... .......... 65% 42.7M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109700K .......... .......... .......... .......... .......... 65% 24.7M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109750K .......... .......... .......... .......... .......... 65% 24.6M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109800K .......... .......... .......... .......... .......... 65% 42.6M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109850K .......... .......... .......... .......... .......... 65% 22.8M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109900K .......... .......... .......... .......... .......... 65% 45.3M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:109950K .......... .......... .......... .......... .......... 65% 24.2M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110000K .......... .......... .......... .......... .......... 65% 36.1M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110050K .......... .......... .......... .......... .......... 65% 24.8M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110100K .......... .......... .......... .......... .......... 65% 25.4M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110150K .......... .......... .......... .......... .......... 65% 44.2M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110200K .......... .......... .......... .......... .......... 65% 24.6M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110250K .......... .......... .......... .......... .......... 65% 24.9M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110300K .......... .......... .......... .......... .......... 65% 35.3M 3s 2024-03-29T04:10:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:110350K .......... .......... .......... .......... .......... 65% 24.7M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110400K .......... .......... .......... .......... .......... 65% 49.0M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110450K .......... .......... .......... .......... .......... 65% 24.8M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110500K .......... .......... .......... .......... .......... 65% 23.6M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110550K .......... .......... .......... .......... .......... 65% 36.5M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110600K .......... .......... .......... .......... .......... 65% 24.2M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110650K .......... .......... .......... .......... .......... 65% 52.2M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110700K .......... .......... .......... .......... .......... 65% 24.1M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110750K .......... .......... .......... .......... .......... 65% 34.3M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110800K .......... .......... .......... .......... .......... 65% 24.5M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110850K .......... .......... .......... .......... .......... 65% 24.7M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110900K .......... .......... .......... .......... .......... 65% 51.7M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:110950K .......... .......... .......... .......... .......... 65% 24.7M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:111000K .......... .......... .......... .......... .......... 65% 29.3M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:111050K .......... .......... .......... .......... .......... 65% 30.9M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:111100K .......... .......... .......... .......... .......... 65% 23.5M 3s 2024-03-29T04:10:41.972 INFO:teuthology.orchestra.run.smithi181.stderr:111150K .......... .......... .......... .......... .......... 65% 23.0M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111200K .......... .......... .......... .......... .......... 65% 50.2M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111250K .......... .......... .......... .......... .......... 66% 25.5M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111300K .......... .......... .......... .......... .......... 66% 30.3M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111350K .......... .......... .......... .......... .......... 66% 24.7M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111400K .......... .......... .......... .......... .......... 66% 26.5M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111450K .......... .......... .......... .......... .......... 66% 40.0M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111500K .......... .......... .......... .......... .......... 66% 25.3M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111550K .......... .......... .......... .......... .......... 66% 34.8M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111600K .......... .......... .......... .......... .......... 66% 24.5M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111650K .......... .......... .......... .......... .......... 66% 23.5M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111700K .......... .......... .......... .......... .......... 66% 48.0M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111750K .......... .......... .......... .......... .......... 66% 24.4M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111800K .......... .......... .......... .......... .......... 66% 38.7M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111850K .......... .......... .......... .......... .......... 66% 23.3M 3s 2024-03-29T04:10:41.973 INFO:teuthology.orchestra.run.smithi181.stderr:111900K .......... .......... .......... .......... .......... 66% 24.9M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:111950K .......... .......... .......... .......... .......... 66% 46.0M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112000K .......... .......... .......... .......... .......... 66% 23.4M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112050K .......... .......... .......... .......... .......... 66% 35.4M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112100K .......... .......... .......... .......... .......... 66% 31.3M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112150K .......... .......... .......... .......... .......... 66% 21.2M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112200K .......... .......... .......... .......... .......... 66% 38.4M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112250K .......... .......... .......... .......... .......... 66% 25.5M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112300K .......... .......... .......... .......... .......... 66% 35.5M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112350K .......... .......... .......... .......... .......... 66% 24.0M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112400K .......... .......... .......... .......... .......... 66% 24.5M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112450K .......... .......... .......... .......... .......... 66% 45.9M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112500K .......... .......... .......... .......... .......... 66% 24.5M 3s 2024-03-29T04:10:41.974 INFO:teuthology.orchestra.run.smithi181.stderr:112550K .......... .......... .......... .......... .......... 66% 36.8M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112600K .......... .......... .......... .......... .......... 66% 28.0M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112650K .......... .......... .......... .......... .......... 66% 21.8M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112700K .......... .......... .......... .......... .......... 66% 46.9M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112750K .......... .......... .......... .......... .......... 66% 24.7M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112800K .......... .......... .......... .......... .......... 66% 35.5M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112850K .......... .......... .......... .......... .......... 66% 24.1M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112900K .......... .......... .......... .......... .......... 67% 46.7M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:112950K .......... .......... .......... .......... .......... 67% 18.1M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113000K .......... .......... .......... .......... .......... 67% 77.5M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113050K .......... .......... .......... .......... .......... 67% 24.2M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113100K .......... .......... .......... .......... .......... 67% 25.2M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113150K .......... .......... .......... .......... .......... 67% 24.8M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113200K .......... .......... .......... .......... .......... 67% 47.6M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113250K .......... .......... .......... .......... .......... 67% 24.6M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113300K .......... .......... .......... .......... .......... 67% 34.1M 3s 2024-03-29T04:10:41.975 INFO:teuthology.orchestra.run.smithi181.stderr:113350K .......... .......... .......... .......... .......... 67% 24.5M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113400K .......... .......... .......... .......... .......... 67% 24.4M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113450K .......... .......... .......... .......... .......... 67% 57.7M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113500K .......... .......... .......... .......... .......... 67% 22.7M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113550K .......... .......... .......... .......... .......... 67% 31.4M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113600K .......... .......... .......... .......... .......... 67% 26.7M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113650K .......... .......... .......... .......... .......... 67% 24.6M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113700K .......... .......... .......... .......... .......... 67% 50.7M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113750K .......... .......... .......... .......... .......... 67% 24.5M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113800K .......... .......... .......... .......... .......... 67% 33.0M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113850K .......... .......... .......... .......... .......... 67% 22.2M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113900K .......... .......... .......... .......... .......... 67% 28.3M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:113950K .......... .......... .......... .......... .......... 67% 49.9M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:114000K .......... .......... .......... .......... .......... 67% 33.2M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:114050K .......... .......... .......... .......... .......... 67% 24.8M 3s 2024-03-29T04:10:41.976 INFO:teuthology.orchestra.run.smithi181.stderr:114100K .......... .......... .......... .......... .......... 67% 25.3M 3s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114150K .......... .......... .......... .......... .......... 67% 25.3M 3s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114200K .......... .......... .......... .......... .......... 67% 41.4M 3s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114250K .......... .......... .......... .......... .......... 67% 31.1M 3s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114300K .......... .......... .......... .......... .......... 67% 27.7M 3s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114350K .......... .......... .......... .......... .......... 67% 24.9M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114400K .......... .......... .......... .......... .......... 67% 24.3M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114450K .......... .......... .......... .......... .......... 67% 54.9M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114500K .......... .......... .......... .......... .......... 67% 23.0M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114550K .......... .......... .......... .......... .......... 67% 36.9M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114600K .......... .......... .......... .......... .......... 68% 20.7M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114650K .......... .......... .......... .......... .......... 68% 34.0M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114700K .......... .......... .......... .......... .......... 68% 43.8M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114750K .......... .......... .......... .......... .......... 68% 28.4M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114800K .......... .......... .......... .......... .......... 68% 24.4M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114850K .......... .......... .......... .......... .......... 68% 28.6M 2s 2024-03-29T04:10:41.977 INFO:teuthology.orchestra.run.smithi181.stderr:114900K .......... .......... .......... .......... .......... 68% 24.6M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:114950K .......... .......... .......... .......... .......... 68% 50.3M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115000K .......... .......... .......... .......... .......... 68% 26.6M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115050K .......... .......... .......... .......... .......... 68% 25.7M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115100K .......... .......... .......... .......... .......... 68% 28.8M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115150K .......... .......... .......... .......... .......... 68% 24.5M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115200K .......... .......... .......... .......... .......... 68% 51.5M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115250K .......... .......... .......... .......... .......... 68% 27.0M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115300K .......... .......... .......... .......... .......... 68% 22.3M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115350K .......... .......... .......... .......... .......... 68% 34.6M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115400K .......... .......... .......... .......... .......... 68% 24.4M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115450K .......... .......... .......... .......... .......... 68% 54.1M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115500K .......... .......... .......... .......... .......... 68% 25.8M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115550K .......... .......... .......... .......... .......... 68% 23.6M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115600K .......... .......... .......... .......... .......... 68% 31.9M 2s 2024-03-29T04:10:41.978 INFO:teuthology.orchestra.run.smithi181.stderr:115650K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115700K .......... .......... .......... .......... .......... 68% 53.9M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115750K .......... .......... .......... .......... .......... 68% 25.2M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115800K .......... .......... .......... .......... .......... 68% 24.8M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115850K .......... .......... .......... .......... .......... 68% 31.1M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115900K .......... .......... .......... .......... .......... 68% 34.6M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:115950K .......... .......... .......... .......... .......... 68% 32.2M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116000K .......... .......... .......... .......... .......... 68% 26.1M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116050K .......... .......... .......... .......... .......... 68% 23.8M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116100K .......... .......... .......... .......... .......... 68% 32.5M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116150K .......... .......... .......... .......... .......... 68% 24.6M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116200K .......... .......... .......... .......... .......... 68% 48.9M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116250K .......... .......... .......... .......... .......... 68% 23.8M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116300K .......... .......... .......... .......... .......... 69% 28.3M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116350K .......... .......... .......... .......... .......... 69% 29.4M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116400K .......... .......... .......... .......... .......... 69% 21.0M 2s 2024-03-29T04:10:41.979 INFO:teuthology.orchestra.run.smithi181.stderr:116450K .......... .......... .......... .......... .......... 69% 27.1M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116500K .......... .......... .......... .......... .......... 69% 52.0M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116550K .......... .......... .......... .......... .......... 69% 24.4M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116600K .......... .......... .......... .......... .......... 69% 34.6M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116650K .......... .......... .......... .......... .......... 69% 23.4M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116700K .......... .......... .......... .......... .......... 69% 25.4M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116750K .......... .......... .......... .......... .......... 69% 45.8M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116800K .......... .......... .......... .......... .......... 69% 24.8M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116850K .......... .......... .......... .......... .......... 69% 33.9M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116900K .......... .......... .......... .......... .......... 69% 24.8M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:116950K .......... .......... .......... .......... .......... 69% 24.8M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:117000K .......... .......... .......... .......... .......... 69% 45.5M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:117050K .......... .......... .......... .......... .......... 69% 27.8M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:117100K .......... .......... .......... .......... .......... 69% 30.9M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:117150K .......... .......... .......... .......... .......... 69% 24.6M 2s 2024-03-29T04:10:41.980 INFO:teuthology.orchestra.run.smithi181.stderr:117200K .......... .......... .......... .......... .......... 69% 23.2M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117250K .......... .......... .......... .......... .......... 69% 48.2M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117300K .......... .......... .......... .......... .......... 69% 24.7M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117350K .......... .......... .......... .......... .......... 69% 35.4M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117400K .......... .......... .......... .......... .......... 69% 24.2M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117450K .......... .......... .......... .......... .......... 69% 23.8M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117500K .......... .......... .......... .......... .......... 69% 52.3M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117550K .......... .......... .......... .......... .......... 69% 24.5M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117600K .......... .......... .......... .......... .......... 69% 33.0M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117650K .......... .......... .......... .......... .......... 69% 24.7M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117700K .......... .......... .......... .......... .......... 69% 51.4M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117750K .......... .......... .......... .......... .......... 69% 24.9M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117800K .......... .......... .......... .......... .......... 69% 24.3M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117850K .......... .......... .......... .......... .......... 69% 28.4M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117900K .......... .......... .......... .......... .......... 69% 27.8M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:117950K .......... .......... .......... .......... .......... 70% 53.8M 2s 2024-03-29T04:10:41.981 INFO:teuthology.orchestra.run.smithi181.stderr:118000K .......... .......... .......... .......... .......... 70% 24.6M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118050K .......... .......... .......... .......... .......... 70% 28.7M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118100K .......... .......... .......... .......... .......... 70% 24.5M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118150K .......... .......... .......... .......... .......... 70% 25.2M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118200K .......... .......... .......... .......... .......... 70% 55.4M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118250K .......... .......... .......... .......... .......... 70% 25.1M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118300K .......... .......... .......... .......... .......... 70% 24.0M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118350K .......... .......... .......... .......... .......... 70% 25.5M 2s 2024-03-29T04:10:41.982 INFO:teuthology.orchestra.run.smithi181.stderr:118400K .......... .......... .......... .......... .......... 70% 26.2M 2s 2024-03-29T04:10:41.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:10:42.053 INFO:teuthology.orchestra.run.smithi181.stderr:118450K .......... .......... .......... .......... .......... 70% 73.3M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118500K .......... .......... .......... .......... .......... 70% 26.2M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118550K .......... .......... .......... .......... .......... 70% 24.6M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118600K .......... .......... .......... .......... .......... 70% 30.4M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118650K .......... .......... .......... .......... .......... 70% 22.6M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118700K .......... .......... .......... .......... .......... 70% 38.7M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118750K .......... .......... .......... .......... .......... 70% 34.3M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118800K .......... .......... .......... .......... .......... 70% 24.1M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118850K .......... .......... .......... .......... .......... 70% 30.1M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118900K .......... .......... .......... .......... .......... 70% 36.6M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:118950K .......... .......... .......... .......... .......... 70% 28.7M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:119000K .......... .......... .......... .......... .......... 70% 29.0M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:119050K .......... .......... .......... .......... .......... 70% 26.8M 2s 2024-03-29T04:10:42.054 INFO:teuthology.orchestra.run.smithi181.stderr:119100K .......... .......... .......... .......... .......... 70% 25.8M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119150K .......... .......... .......... .......... .......... 70% 24.2M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119200K .......... .......... .......... .......... .......... 70% 55.4M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119250K .......... .......... .......... .......... .......... 70% 27.2M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119300K .......... .......... .......... .......... .......... 70% 29.5M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119350K .......... .......... .......... .......... .......... 70% 23.9M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119400K .......... .......... .......... .......... .......... 70% 23.7M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119450K .......... .......... .......... .......... .......... 70% 33.1M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119500K .......... .......... .......... .......... .......... 70% 38.5M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119550K .......... .......... .......... .......... .......... 70% 29.8M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119600K .......... .......... .......... .......... .......... 70% 23.6M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119650K .......... .......... .......... .......... .......... 71% 25.5M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119700K .......... .......... .......... .......... .......... 71% 43.2M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119750K .......... .......... .......... .......... .......... 71% 34.4M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119800K .......... .......... .......... .......... .......... 71% 25.6M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119850K .......... .......... .......... .......... .......... 71% 25.1M 2s 2024-03-29T04:10:42.055 INFO:teuthology.orchestra.run.smithi181.stderr:119900K .......... .......... .......... .......... .......... 71% 23.1M 2s 2024-03-29T04:10:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:119950K .......... .......... .......... .......... .......... 71% 48.8M 2s 2024-03-29T04:10:42.056 INFO:teuthology.orchestra.run.smithi181.stderr:120000K .......... .......... .......... .......... .......... 71% 34.2M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120050K .......... .......... .......... .......... .......... 71% 722K 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120100K .......... .......... .......... .......... .......... 71% 368M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120150K .......... .......... .......... .......... .......... 71% 417M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120200K .......... .......... .......... .......... .......... 71% 365M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120250K .......... .......... .......... .......... .......... 71% 418M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120300K .......... .......... .......... .......... .......... 71% 411M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120350K .......... .......... .......... .......... .......... 71% 431M 2s 2024-03-29T04:10:43.953 INFO:teuthology.orchestra.run.smithi181.stderr:120400K .......... .......... .......... .......... .......... 71% 338M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120450K .......... .......... .......... .......... .......... 71% 423M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120500K .......... .......... .......... .......... .......... 71% 418M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120550K .......... .......... .......... .......... .......... 71% 405M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120600K .......... .......... .......... .......... .......... 71% 355M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120650K .......... .......... .......... .......... .......... 71% 420M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120700K .......... .......... .......... .......... .......... 71% 416M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120750K .......... .......... .......... .......... .......... 71% 409M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120800K .......... .......... .......... .......... .......... 71% 326M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120850K .......... .......... .......... .......... .......... 71% 398M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120900K .......... .......... .......... .......... .......... 71% 383M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:120950K .......... .......... .......... .......... .......... 71% 394M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:121000K .......... .......... .......... .......... .......... 71% 368M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:121050K .......... .......... .......... .......... .......... 71% 378M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:121100K .......... .......... .......... .......... .......... 71% 289M 2s 2024-03-29T04:10:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:121150K .......... .......... .......... .......... .......... 71% 221M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121200K .......... .......... .......... .......... .......... 71% 319M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121250K .......... .......... .......... .......... .......... 71% 542M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121300K .......... .......... .......... .......... .......... 71% 621M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121350K .......... .......... .......... .......... .......... 72% 649M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121400K .......... .......... .......... .......... .......... 72% 545M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121450K .......... .......... .......... .......... .......... 72% 602M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121500K .......... .......... .......... .......... .......... 72% 624M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121550K .......... .......... .......... .......... .......... 72% 623M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121600K .......... .......... .......... .......... .......... 72% 547M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121650K .......... .......... .......... .......... .......... 72% 590M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121700K .......... .......... .......... .......... .......... 72% 660M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121750K .......... .......... .......... .......... .......... 72% 686M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121800K .......... .......... .......... .......... .......... 72% 563M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121850K .......... .......... .......... .......... .......... 72% 681M 2s 2024-03-29T04:10:43.955 INFO:teuthology.orchestra.run.smithi181.stderr:121900K .......... .......... .......... .......... .......... 72% 676M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:121950K .......... .......... .......... .......... .......... 72% 681M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122000K .......... .......... .......... .......... .......... 72% 481M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122050K .......... .......... .......... .......... .......... 72% 396M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122100K .......... .......... .......... .......... .......... 72% 653M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122150K .......... .......... .......... .......... .......... 72% 711M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122200K .......... .......... .......... .......... .......... 72% 44.1M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122250K .......... .......... .......... .......... .......... 72% 25.1M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122300K .......... .......... .......... .......... .......... 72% 49.6M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122350K .......... .......... .......... .......... .......... 72% 27.9M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122400K .......... .......... .......... .......... .......... 72% 29.6M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122450K .......... .......... .......... .......... .......... 72% 24.6M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122500K .......... .......... .......... .......... .......... 72% 24.8M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122550K .......... .......... .......... .......... .......... 72% 40.7M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122600K .......... .......... .......... .......... .......... 72% 24.6M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122650K .......... .......... .......... .......... .......... 72% 37.8M 2s 2024-03-29T04:10:43.956 INFO:teuthology.orchestra.run.smithi181.stderr:122700K .......... .......... .......... .......... .......... 72% 24.2M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:122750K .......... .......... .......... .......... .......... 72% 23.9M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:122800K .......... .......... .......... .......... .......... 72% 41.9M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:122850K .......... .......... .......... .......... .......... 72% 25.9M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:122900K .......... .......... .......... .......... .......... 72% 39.5M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:122950K .......... .......... .......... .......... .......... 72% 24.7M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123000K .......... .......... .......... .......... .......... 72% 24.9M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123050K .......... .......... .......... .......... .......... 73% 43.6M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123100K .......... .......... .......... .......... .......... 73% 24.5M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123150K .......... .......... .......... .......... .......... 73% 38.7M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123200K .......... .......... .......... .......... .......... 73% 24.1M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123250K .......... .......... .......... .......... .......... 73% 20.3M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123300K .......... .......... .......... .......... .......... 73% 62.8M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123350K .......... .......... .......... .......... .......... 73% 24.3M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123400K .......... .......... .......... .......... .......... 73% 37.0M 2s 2024-03-29T04:10:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:123450K .......... .......... .......... .......... .......... 73% 26.6M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123500K .......... .......... .......... .......... .......... 73% 24.1M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123550K .......... .......... .......... .......... .......... 73% 41.4M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123600K .......... .......... .......... .......... .......... 73% 25.5M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123650K .......... .......... .......... .......... .......... 73% 37.0M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123700K .......... .......... .......... .......... .......... 73% 25.5M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123750K .......... .......... .......... .......... .......... 73% 24.3M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123800K .......... .......... .......... .......... .......... 73% 43.1M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123850K .......... .......... .......... .......... .......... 73% 24.4M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123900K .......... .......... .......... .......... .......... 73% 38.7M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:123950K .......... .......... .......... .......... .......... 73% 24.7M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:124000K .......... .......... .......... .......... .......... 73% 24.6M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:124050K .......... .......... .......... .......... .......... 73% 42.7M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:124100K .......... .......... .......... .......... .......... 73% 23.4M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:124150K .......... .......... .......... .......... .......... 73% 38.6M 2s 2024-03-29T04:10:43.958 INFO:teuthology.orchestra.run.smithi181.stderr:124200K .......... .......... .......... .......... .......... 73% 25.0M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124250K .......... .......... .......... .......... .......... 73% 24.6M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124300K .......... .......... .......... .......... .......... 73% 41.3M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124350K .......... .......... .......... .......... .......... 73% 24.8M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124400K .......... .......... .......... .......... .......... 73% 38.5M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124450K .......... .......... .......... .......... .......... 73% 24.7M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124500K .......... .......... .......... .......... .......... 73% 24.6M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124550K .......... .......... .......... .......... .......... 73% 42.1M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124600K .......... .......... .......... .......... .......... 73% 32.1M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124650K .......... .......... .......... .......... .......... 73% 27.6M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124700K .......... .......... .......... .......... .......... 74% 24.6M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124750K .......... .......... .......... .......... .......... 74% 24.7M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124800K .......... .......... .......... .......... .......... 74% 49.9M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124850K .......... .......... .......... .......... .......... 74% 29.1M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124900K .......... .......... .......... .......... .......... 74% 27.4M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:124950K .......... .......... .......... .......... .......... 74% 23.5M 2s 2024-03-29T04:10:43.959 INFO:teuthology.orchestra.run.smithi181.stderr:125000K .......... .......... .......... .......... .......... 74% 22.7M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125050K .......... .......... .......... .......... .......... 74% 65.0M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125100K .......... .......... .......... .......... .......... 74% 30.0M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125150K .......... .......... .......... .......... .......... 74% 22.7M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125200K .......... .......... .......... .......... .......... 74% 33.2M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125250K .......... .......... .......... .......... .......... 74% 24.5M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125300K .......... .......... .......... .......... .......... 74% 37.4M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125350K .......... .......... .......... .......... .......... 74% 24.7M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125400K .......... .......... .......... .......... .......... 74% 31.4M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125450K .......... .......... .......... .......... .......... 74% 20.0M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125500K .......... .......... .......... .......... .......... 74% 29.7M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125550K .......... .......... .......... .......... .......... 74% 52.8M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125600K .......... .......... .......... .......... .......... 74% 32.0M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125650K .......... .......... .......... .......... .......... 74% 24.0M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125700K .......... .......... .......... .......... .......... 74% 26.7M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125750K .......... .......... .......... .......... .......... 74% 24.6M 2s 2024-03-29T04:10:43.960 INFO:teuthology.orchestra.run.smithi181.stderr:125800K .......... .......... .......... .......... .......... 74% 51.5M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:125850K .......... .......... .......... .......... .......... 74% 27.3M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:125900K .......... .......... .......... .......... .......... 74% 26.4M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:125950K .......... .......... .......... .......... .......... 74% 25.8M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126000K .......... .......... .......... .......... .......... 74% 27.5M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126050K .......... .......... .......... .......... .......... 74% 50.3M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126100K .......... .......... .......... .......... .......... 74% 27.0M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126150K .......... .......... .......... .......... .......... 74% 25.0M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126200K .......... .......... .......... .......... .......... 74% 31.4M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126250K .......... .......... .......... .......... .......... 74% 20.4M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126300K .......... .......... .......... .......... .......... 74% 70.7M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126350K .......... .......... .......... .......... .......... 74% 25.5M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126400K .......... .......... .......... .......... .......... 75% 26.4M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126450K .......... .......... .......... .......... .......... 75% 35.6M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126500K .......... .......... .......... .......... .......... 75% 24.0M 2s 2024-03-29T04:10:43.961 INFO:teuthology.orchestra.run.smithi181.stderr:126550K .......... .......... .......... .......... .......... 75% 42.8M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126600K .......... .......... .......... .......... .......... 75% 24.8M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126650K .......... .......... .......... .......... .......... 75% 27.0M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126700K .......... .......... .......... .......... .......... 75% 26.2M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126750K .......... .......... .......... .......... .......... 75% 27.1M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126800K .......... .......... .......... .......... .......... 75% 25.2M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126850K .......... .......... .......... .......... .......... 75% 49.0M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126900K .......... .......... .......... .......... .......... 75% 25.3M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:126950K .......... .......... .......... .......... .......... 75% 35.4M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127000K .......... .......... .......... .......... .......... 75% 21.8M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127050K .......... .......... .......... .......... .......... 75% 24.9M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127100K .......... .......... .......... .......... .......... 75% 62.1M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127150K .......... .......... .......... .......... .......... 75% 23.8M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127200K .......... .......... .......... .......... .......... 75% 31.1M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127250K .......... .......... .......... .......... .......... 75% 23.7M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127300K .......... .......... .......... .......... .......... 75% 23.1M 2s 2024-03-29T04:10:43.962 INFO:teuthology.orchestra.run.smithi181.stderr:127350K .......... .......... .......... .......... .......... 75% 57.2M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127400K .......... .......... .......... .......... .......... 75% 24.1M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127450K .......... .......... .......... .......... .......... 75% 31.3M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127500K .......... .......... .......... .......... .......... 75% 24.9M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127550K .......... .......... .......... .......... .......... 75% 23.6M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127600K .......... .......... .......... .......... .......... 75% 54.6M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127650K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127700K .......... .......... .......... .......... .......... 75% 22.0M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127750K .......... .......... .......... .......... .......... 75% 43.5M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127800K .......... .......... .......... .......... .......... 75% 9.85M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127850K .......... .......... .......... .......... .......... 75% 219M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127900K .......... .......... .......... .......... .......... 75% 167M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:127950K .......... .......... .......... .......... .......... 75% 33.0M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:128000K .......... .......... .......... .......... .......... 75% 30.3M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:128050K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:128100K .......... .......... .......... .......... .......... 76% 47.0M 2s 2024-03-29T04:10:43.963 INFO:teuthology.orchestra.run.smithi181.stderr:128150K .......... .......... .......... .......... .......... 76% 24.8M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128200K .......... .......... .......... .......... .......... 76% 34.3M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128250K .......... .......... .......... .......... .......... 76% 24.2M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128300K .......... .......... .......... .......... .......... 76% 23.3M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128350K .......... .......... .......... .......... .......... 76% 49.7M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128400K .......... .......... .......... .......... .......... 76% 23.9M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128450K .......... .......... .......... .......... .......... 76% 38.0M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128500K .......... .......... .......... .......... .......... 76% 23.5M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128550K .......... .......... .......... .......... .......... 76% 23.7M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128600K .......... .......... .......... .......... .......... 76% 46.4M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128650K .......... .......... .......... .......... .......... 76% 25.0M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128700K .......... .......... .......... .......... .......... 76% 36.6M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128750K .......... .......... .......... .......... .......... 76% 25.0M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128800K .......... .......... .......... .......... .......... 76% 30.5M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128850K .......... .......... .......... .......... .......... 76% 34.7M 2s 2024-03-29T04:10:43.964 INFO:teuthology.orchestra.run.smithi181.stderr:128900K .......... .......... .......... .......... .......... 76% 23.6M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:128950K .......... .......... .......... .......... .......... 76% 24.1M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129000K .......... .......... .......... .......... .......... 76% 36.2M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129050K .......... .......... .......... .......... .......... 76% 25.4M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129100K .......... .......... .......... .......... .......... 76% 45.0M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129150K .......... .......... .......... .......... .......... 76% 24.8M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129200K .......... .......... .......... .......... .......... 76% 34.6M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129250K .......... .......... .......... .......... .......... 76% 28.2M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129300K .......... .......... .......... .......... .......... 76% 34.5M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129350K .......... .......... .......... .......... .......... 76% 28.5M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129400K .......... .......... .......... .......... .......... 76% 23.1M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129450K .......... .......... .......... .......... .......... 76% 39.9M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129500K .......... .......... .......... .......... .......... 76% 19.6M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129550K .......... .......... .......... .......... .......... 76% 27.3M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129600K .......... .......... .......... .......... .......... 76% 57.0M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129650K .......... .......... .......... .......... .......... 76% 24.4M 2s 2024-03-29T04:10:43.965 INFO:teuthology.orchestra.run.smithi181.stderr:129700K .......... .......... .......... .......... .......... 76% 32.4M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:129750K .......... .......... .......... .......... .......... 77% 22.6M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:129800K .......... .......... .......... .......... .......... 77% 24.7M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:129850K .......... .......... .......... .......... .......... 77% 54.6M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:129900K .......... .......... .......... .......... .......... 77% 24.2M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:129950K .......... .......... .......... .......... .......... 77% 33.0M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130000K .......... .......... .......... .......... .......... 77% 24.6M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130050K .......... .......... .......... .......... .......... 77% 24.2M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130100K .......... .......... .......... .......... .......... 77% 60.9M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130150K .......... .......... .......... .......... .......... 77% 24.3M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130200K .......... .......... .......... .......... .......... 77% 39.9M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130250K .......... .......... .......... .......... .......... 77% 23.0M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130300K .......... .......... .......... .......... .......... 77% 24.0M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130350K .......... .......... .......... .......... .......... 77% 47.2M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130400K .......... .......... .......... .......... .......... 77% 24.9M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130450K .......... .......... .......... .......... .......... 77% 24.0M 2s 2024-03-29T04:10:43.966 INFO:teuthology.orchestra.run.smithi181.stderr:130500K .......... .......... .......... .......... .......... 77% 32.2M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130550K .......... .......... .......... .......... .......... 77% 34.0M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130600K .......... .......... .......... .......... .......... 77% 36.1M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130650K .......... .......... .......... .......... .......... 77% 25.0M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130700K .......... .......... .......... .......... .......... 77% 29.4M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130750K .......... .......... .......... .......... .......... 77% 34.5M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130800K .......... .......... .......... .......... .......... 77% 31.8M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130850K .......... .......... .......... .......... .......... 77% 27.8M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130900K .......... .......... .......... .......... .......... 77% 25.9M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:130950K .......... .......... .......... .......... .......... 77% 22.1M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131000K .......... .......... .......... .......... .......... 77% 30.9M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131050K .......... .......... .......... .......... .......... 77% 49.9M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131100K .......... .......... .......... .......... .......... 77% 28.3M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131150K .......... .......... .......... .......... .......... 77% 24.5M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131200K .......... .......... .......... .......... .......... 77% 25.1M 2s 2024-03-29T04:10:43.967 INFO:teuthology.orchestra.run.smithi181.stderr:131250K .......... .......... .......... .......... .......... 77% 25.2M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131300K .......... .......... .......... .......... .......... 77% 30.3M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131350K .......... .......... .......... .......... .......... 77% 44.9M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131400K .......... .......... .......... .......... .......... 77% 26.1M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131450K .......... .......... .......... .......... .......... 78% 29.4M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131500K .......... .......... .......... .......... .......... 78% 23.1M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131550K .......... .......... .......... .......... .......... 78% 46.6M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131600K .......... .......... .......... .......... .......... 78% 31.6M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131650K .......... .......... .......... .......... .......... 78% 25.2M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131700K .......... .......... .......... .......... .......... 78% 25.3M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131750K .......... .......... .......... .......... .......... 78% 31.9M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131800K .......... .......... .......... .......... .......... 78% 26.0M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131850K .......... .......... .......... .......... .......... 78% 39.4M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131900K .......... .......... .......... .......... .......... 78% 27.0M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:131950K .......... .......... .......... .......... .......... 78% 26.2M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:132000K .......... .......... .......... .......... .......... 78% 32.6M 2s 2024-03-29T04:10:43.968 INFO:teuthology.orchestra.run.smithi181.stderr:132050K .......... .......... .......... .......... .......... 78% 28.6M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132100K .......... .......... .......... .......... .......... 78% 23.2M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132150K .......... .......... .......... .......... .......... 78% 36.4M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132200K .......... .......... .......... .......... .......... 78% 24.5M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132250K .......... .......... .......... .......... .......... 78% 24.6M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132300K .......... .......... .......... .......... .......... 78% 44.2M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132350K .......... .......... .......... .......... .......... 78% 23.8M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132400K .......... .......... .......... .......... .......... 78% 37.0M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132450K .......... .......... .......... .......... .......... 78% 24.5M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132500K .......... .......... .......... .......... .......... 78% 37.2M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132550K .......... .......... .......... .......... .......... 78% 27.1M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132600K .......... .......... .......... .......... .......... 78% 22.7M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132650K .......... .......... .......... .......... .......... 78% 40.7M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132700K .......... .......... .......... .......... .......... 78% 26.2M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132750K .......... .......... .......... .......... .......... 78% 43.6M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132800K .......... .......... .......... .......... .......... 78% 25.9M 2s 2024-03-29T04:10:43.969 INFO:teuthology.orchestra.run.smithi181.stderr:132850K .......... .......... .......... .......... .......... 78% 20.3M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:132900K .......... .......... .......... .......... .......... 78% 48.8M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:132950K .......... .......... .......... .......... .......... 78% 30.0M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133000K .......... .......... .......... .......... .......... 78% 30.6M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133050K .......... .......... .......... .......... .......... 78% 27.4M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133100K .......... .......... .......... .......... .......... 78% 20.9M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133150K .......... .......... .......... .......... .......... 79% 45.7M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133200K .......... .......... .......... .......... .......... 79% 23.9M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133250K .......... .......... .......... .......... .......... 79% 32.3M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133300K .......... .......... .......... .......... .......... 79% 28.0M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133350K .......... .......... .......... .......... .......... 79% 26.0M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133400K .......... .......... .......... .......... .......... 79% 39.1M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133450K .......... .......... .......... .......... .......... 79% 29.8M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133500K .......... .......... .......... .......... .......... 79% 28.9M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133550K .......... .......... .......... .......... .......... 79% 23.8M 2s 2024-03-29T04:10:43.970 INFO:teuthology.orchestra.run.smithi181.stderr:133600K .......... .......... .......... .......... .......... 79% 24.4M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133650K .......... .......... .......... .......... .......... 79% 49.8M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133700K .......... .......... .......... .......... .......... 79% 32.6M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133750K .......... .......... .......... .......... .......... 79% 25.7M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133800K .......... .......... .......... .......... .......... 79% 28.5M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133850K .......... .......... .......... .......... .......... 79% 20.7M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133900K .......... .......... .......... .......... .......... 79% 48.9M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:133950K .......... .......... .......... .......... .......... 79% 23.7M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134000K .......... .......... .......... .......... .......... 79% 34.1M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134050K .......... .......... .......... .......... .......... 79% 25.6M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134100K .......... .......... .......... .......... .......... 79% 24.4M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134150K .......... .......... .......... .......... .......... 79% 53.3M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134200K .......... .......... .......... .......... .......... 79% 29.4M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134250K .......... .......... .......... .......... .......... 79% 20.0M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134300K .......... .......... .......... .......... .......... 79% 43.6M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134350K .......... .......... .......... .......... .......... 79% 21.0M 2s 2024-03-29T04:10:43.971 INFO:teuthology.orchestra.run.smithi181.stderr:134400K .......... .......... .......... .......... .......... 79% 49.0M 2s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134450K .......... .......... .......... .......... .......... 79% 25.3M 2s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134500K .......... .......... .......... .......... .......... 79% 24.2M 2s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134550K .......... .......... .......... .......... .......... 79% 31.5M 2s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134600K .......... .......... .......... .......... .......... 79% 26.9M 2s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134650K .......... .......... .......... .......... .......... 79% 55.0M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134700K .......... .......... .......... .......... .......... 79% 26.1M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134750K .......... .......... .......... .......... .......... 79% 29.8M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134800K .......... .......... .......... .......... .......... 79% 24.1M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134850K .......... .......... .......... .......... .......... 80% 25.1M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134900K .......... .......... .......... .......... .......... 80% 56.5M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:134950K .......... .......... .......... .......... .......... 80% 21.2M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:135000K .......... .......... .......... .......... .......... 80% 39.1M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:135050K .......... .......... .......... .......... .......... 80% 24.5M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:135100K .......... .......... .......... .......... .......... 80% 25.3M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:135150K .......... .......... .......... .......... .......... 80% 41.1M 1s 2024-03-29T04:10:43.972 INFO:teuthology.orchestra.run.smithi181.stderr:135200K .......... .......... .......... .......... .......... 80% 24.4M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135250K .......... .......... .......... .......... .......... 80% 31.1M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135300K .......... .......... .......... .......... .......... 80% 25.1M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135350K .......... .......... .......... .......... .......... 80% 25.0M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135400K .......... .......... .......... .......... .......... 80% 55.3M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135450K .......... .......... .......... .......... .......... 80% 23.4M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135500K .......... .......... .......... .......... .......... 80% 39.8M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135550K .......... .......... .......... .......... .......... 80% 24.5M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135600K .......... .......... .......... .......... .......... 80% 23.8M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135650K .......... .......... .......... .......... .......... 80% 41.2M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135700K .......... .......... .......... .......... .......... 80% 24.9M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135750K .......... .......... .......... .......... .......... 80% 26.3M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135800K .......... .......... .......... .......... .......... 80% 34.2M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135850K .......... .......... .......... .......... .......... 80% 25.6M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135900K .......... .......... .......... .......... .......... 80% 44.3M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:135950K .......... .......... .......... .......... .......... 80% 24.2M 1s 2024-03-29T04:10:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:136000K .......... .......... .......... .......... .......... 80% 36.4M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136050K .......... .......... .......... .......... .......... 80% 24.8M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136100K .......... .......... .......... .......... .......... 80% 25.0M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136150K .......... .......... .......... .......... .......... 80% 46.4M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136200K .......... .......... .......... .......... .......... 80% 24.5M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136250K .......... .......... .......... .......... .......... 80% 30.9M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136300K .......... .......... .......... .......... .......... 80% 26.9M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136350K .......... .......... .......... .......... .......... 80% 22.4M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136400K .......... .......... .......... .......... .......... 80% 59.8M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136450K .......... .......... .......... .......... .......... 80% 27.1M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136500K .......... .......... .......... .......... .......... 81% 30.6M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136550K .......... .......... .......... .......... .......... 81% 25.4M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136600K .......... .......... .......... .......... .......... 81% 19.2M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136650K .......... .......... .......... .......... .......... 81% 31.8M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136700K .......... .......... .......... .......... .......... 81% 18.3M 1s 2024-03-29T04:10:43.974 INFO:teuthology.orchestra.run.smithi181.stderr:136750K .......... .......... .......... .......... .......... 81% 100M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:136800K .......... .......... .......... .......... .......... 81% 37.4M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:136850K .......... .......... .......... .......... .......... 81% 32.7M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:136900K .......... .......... .......... .......... .......... 81% 34.4M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:136950K .......... .......... .......... .......... .......... 81% 23.8M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137000K .......... .......... .......... .......... .......... 81% 29.3M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137050K .......... .......... .......... .......... .......... 81% 28.7M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137100K .......... .......... .......... .......... .......... 81% 24.6M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137150K .......... .......... .......... .......... .......... 81% 51.7M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137200K .......... .......... .......... .......... .......... 81% 24.1M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137250K .......... .......... .......... .......... .......... 81% 34.3M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137300K .......... .......... .......... .......... .......... 81% 24.7M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137350K .......... .......... .......... .......... .......... 81% 24.3M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137400K .......... .......... .......... .......... .......... 81% 24.1M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137450K .......... .......... .......... .......... .......... 81% 51.0M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137500K .......... .......... .......... .......... .......... 81% 23.3M 1s 2024-03-29T04:10:43.975 INFO:teuthology.orchestra.run.smithi181.stderr:137550K .......... .......... .......... .......... .......... 81% 37.9M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137600K .......... .......... .......... .......... .......... 81% 25.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137650K .......... .......... .......... .......... .......... 81% 23.7M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137700K .......... .......... .......... .......... .......... 81% 50.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137750K .......... .......... .......... .......... .......... 81% 33.2M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137800K .......... .......... .......... .......... .......... 81% 19.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137850K .......... .......... .......... .......... .......... 81% 35.4M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137900K .......... .......... .......... .......... .......... 81% 24.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:137950K .......... .......... .......... .......... .......... 81% 51.2M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138000K .......... .......... .......... .......... .......... 81% 23.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138050K .......... .......... .......... .......... .......... 81% 33.8M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138100K .......... .......... .......... .......... .......... 81% 32.3M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138150K .......... .......... .......... .......... .......... 81% 16.6M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138200K .......... .......... .......... .......... .......... 82% 22.9M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138250K .......... .......... .......... .......... .......... 82% 22.7M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138300K .......... .......... .......... .......... .......... 82% 22.7M 1s 2024-03-29T04:10:43.976 INFO:teuthology.orchestra.run.smithi181.stderr:138350K .......... .......... .......... .......... .......... 82% 35.1M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138400K .......... .......... .......... .......... .......... 82% 179M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138450K .......... .......... .......... .......... .......... 82% 48.4M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138500K .......... .......... .......... .......... .......... 82% 30.6M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138550K .......... .......... .......... .......... .......... 82% 26.7M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138600K .......... .......... .......... .......... .......... 82% 24.0M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138650K .......... .......... .......... .......... .......... 82% 21.0M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138700K .......... .......... .......... .......... .......... 82% 43.2M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138750K .......... .......... .......... .......... .......... 82% 24.1M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138800K .......... .......... .......... .......... .......... 82% 31.4M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138850K .......... .......... .......... .......... .......... 82% 22.1M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138900K .......... .......... .......... .......... .......... 82% 17.7M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:138950K .......... .......... .......... .......... .......... 82% 43.6M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:139000K .......... .......... .......... .......... .......... 82% 17.0M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:139050K .......... .......... .......... .......... .......... 82% 21.8M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:139100K .......... .......... .......... .......... .......... 82% 29.0M 1s 2024-03-29T04:10:43.977 INFO:teuthology.orchestra.run.smithi181.stderr:139150K .......... .......... .......... .......... .......... 82% 20.0M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139200K .......... .......... .......... .......... .......... 82% 30.2M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139250K .......... .......... .......... .......... .......... 82% 28.2M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139300K .......... .......... .......... .......... .......... 82% 56.6M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139350K .......... .......... .......... .......... .......... 82% 29.4M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139400K .......... .......... .......... .......... .......... 82% 69.5M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139450K .......... .......... .......... .......... .......... 82% 90.5M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139500K .......... .......... .......... .......... .......... 82% 47.5M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139550K .......... .......... .......... .......... .......... 82% 37.5M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139600K .......... .......... .......... .......... .......... 82% 20.0M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139650K .......... .......... .......... .......... .......... 82% 33.1M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139700K .......... .......... .......... .......... .......... 82% 51.3M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139750K .......... .......... .......... .......... .......... 82% 25.6M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139800K .......... .......... .......... .......... .......... 82% 31.1M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139850K .......... .......... .......... .......... .......... 82% 24.6M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139900K .......... .......... .......... .......... .......... 83% 24.6M 1s 2024-03-29T04:10:43.978 INFO:teuthology.orchestra.run.smithi181.stderr:139950K .......... .......... .......... .......... .......... 83% 53.6M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140000K .......... .......... .......... .......... .......... 83% 25.2M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140050K .......... .......... .......... .......... .......... 83% 29.1M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140100K .......... .......... .......... .......... .......... 83% 26.2M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140150K .......... .......... .......... .......... .......... 83% 25.3M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140200K .......... .......... .......... .......... .......... 83% 43.9M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140250K .......... .......... .......... .......... .......... 83% 25.6M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140300K .......... .......... .......... .......... .......... 83% 32.0M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140350K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140400K .......... .......... .......... .......... .......... 83% 25.5M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140450K .......... .......... .......... .......... .......... 83% 48.5M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140500K .......... .......... .......... .......... .......... 83% 22.4M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140550K .......... .......... .......... .......... .......... 83% 31.7M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140600K .......... .......... .......... .......... .......... 83% 23.0M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140650K .......... .......... .......... .......... .......... 83% 27.0M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140700K .......... .......... .......... .......... .......... 83% 20.7M 1s 2024-03-29T04:10:43.979 INFO:teuthology.orchestra.run.smithi181.stderr:140750K .......... .......... .......... .......... .......... 83% 18.1M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:140800K .......... .......... .......... .......... .......... 83% 15.2M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:140850K .......... .......... .......... .......... .......... 83% 18.3M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:140900K .......... .......... .......... .......... .......... 83% 12.6M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:140950K .......... .......... .......... .......... .......... 83% 35.0M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141000K .......... .......... .......... .......... .......... 83% 18.2M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141050K .......... .......... .......... .......... .......... 83% 17.9M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141100K .......... .......... .......... .......... .......... 83% 48.6M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141150K .......... .......... .......... .......... .......... 83% 18.1M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141200K .......... .......... .......... .......... .......... 83% 48.8M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141250K .......... .......... .......... .......... .......... 83% 23.1M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141300K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141350K .......... .......... .......... .......... .......... 83% 41.3M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141400K .......... .......... .......... .......... .......... 83% 28.8M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141450K .......... .......... .......... .......... .......... 83% 99.9M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141500K .......... .......... .......... .......... .......... 83% 104M 1s 2024-03-29T04:10:43.980 INFO:teuthology.orchestra.run.smithi181.stderr:141550K .......... .......... .......... .......... .......... 84% 23.0M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141600K .......... .......... .......... .......... .......... 84% 22.8M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141650K .......... .......... .......... .......... .......... 84% 13.2M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141700K .......... .......... .......... .......... .......... 84% 28.9M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141750K .......... .......... .......... .......... .......... 84% 91.4M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141800K .......... .......... .......... .......... .......... 84% 105M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141850K .......... .......... .......... .......... .......... 84% 103M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141900K .......... .......... .......... .......... .......... 84% 111M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:141950K .......... .......... .......... .......... .......... 84% 109M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142000K .......... .......... .......... .......... .......... 84% 45.0M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142050K .......... .......... .......... .......... .......... 84% 103M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142100K .......... .......... .......... .......... .......... 84% 113M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142150K .......... .......... .......... .......... .......... 84% 21.4M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142200K .......... .......... .......... .......... .......... 84% 54.3M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142250K .......... .......... .......... .......... .......... 84% 24.3M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142300K .......... .......... .......... .......... .......... 84% 24.3M 1s 2024-03-29T04:10:43.981 INFO:teuthology.orchestra.run.smithi181.stderr:142350K .......... .......... .......... .......... .......... 84% 34.2M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142400K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142450K .......... .......... .......... .......... .......... 84% 51.3M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142500K .......... .......... .......... .......... .......... 84% 24.6M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142550K .......... .......... .......... .......... .......... 84% 33.1M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142600K .......... .......... .......... .......... .......... 84% 23.7M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142650K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142700K .......... .......... .......... .......... .......... 84% 61.0M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142750K .......... .......... .......... .......... .......... 84% 25.3M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142800K .......... .......... .......... .......... .......... 84% 15.1M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142850K .......... .......... .......... .......... .......... 84% 12.9M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142900K .......... .......... .......... .......... .......... 84% 12.7M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:142950K .......... .......... .......... .......... .......... 84% 30.0M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:143000K .......... .......... .......... .......... .......... 84% 99.7M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:143050K .......... .......... .......... .......... .......... 84% 115M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:143100K .......... .......... .......... .......... .......... 84% 108M 1s 2024-03-29T04:10:43.982 INFO:teuthology.orchestra.run.smithi181.stderr:143150K .......... .......... .......... .......... .......... 84% 80.9M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143200K .......... .......... .......... .......... .......... 84% 90.8M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143250K .......... .......... .......... .......... .......... 85% 27.7M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143300K .......... .......... .......... .......... .......... 85% 20.9M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143350K .......... .......... .......... .......... .......... 85% 48.3M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143400K .......... .......... .......... .......... .......... 85% 22.9M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143450K .......... .......... .......... .......... .......... 85% 37.0M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143500K .......... .......... .......... .......... .......... 85% 24.2M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143550K .......... .......... .......... .......... .......... 85% 41.6M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143600K .......... .......... .......... .......... .......... 85% 25.2M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143650K .......... .......... .......... .......... .......... 85% 24.9M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143700K .......... .......... .......... .......... .......... 85% 35.7M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143750K .......... .......... .......... .......... .......... 85% 25.9M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143800K .......... .......... .......... .......... .......... 85% 24.2M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143850K .......... .......... .......... .......... .......... 85% 40.8M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143900K .......... .......... .......... .......... .......... 85% 25.6M 1s 2024-03-29T04:10:43.983 INFO:teuthology.orchestra.run.smithi181.stderr:143950K .......... .......... .......... .......... .......... 85% 38.4M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144000K .......... .......... .......... .......... .......... 85% 23.8M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144050K .......... .......... .......... .......... .......... 85% 25.0M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144100K .......... .......... .......... .......... .......... 85% 42.0M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144150K .......... .......... .......... .......... .......... 85% 25.0M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144200K .......... .......... .......... .......... .......... 85% 36.2M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144250K .......... .......... .......... .......... .......... 85% 24.7M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144300K .......... .......... .......... .......... .......... 85% 32.5M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144350K .......... .......... .......... .......... .......... 85% 23.5M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144400K .......... .......... .......... .......... .......... 85% 31.9M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144450K .......... .......... .......... .......... .......... 85% 31.9M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144500K .......... .......... .......... .......... .......... 85% 36.9M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144550K .......... .......... .......... .......... .......... 85% 24.8M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144600K .......... .......... .......... .......... .......... 85% 24.1M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144650K .......... .......... .......... .......... .......... 85% 47.0M 1s 2024-03-29T04:10:43.984 INFO:teuthology.orchestra.run.smithi181.stderr:144700K .......... .......... .......... .......... .......... 85% 25.2M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:144750K .......... .......... .......... .......... .......... 85% 35.2M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:144800K .......... .......... .......... .......... .......... 85% 24.7M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:144850K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:144900K .......... .......... .......... .......... .......... 85% 46.9M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:144950K .......... .......... .......... .......... .......... 86% 24.4M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145000K .......... .......... .......... .......... .......... 86% 23.7M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145050K .......... .......... .......... .......... .......... 86% 37.4M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145100K .......... .......... .......... .......... .......... 86% 25.0M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145150K .......... .......... .......... .......... .......... 86% 44.5M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145200K .......... .......... .......... .......... .......... 86% 24.3M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145250K .......... .......... .......... .......... .......... 86% 33.5M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145300K .......... .......... .......... .......... .......... 86% 25.5M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145350K .......... .......... .......... .......... .......... 86% 25.6M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145400K .......... .......... .......... .......... .......... 86% 43.6M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145450K .......... .......... .......... .......... .......... 86% 24.5M 1s 2024-03-29T04:10:43.985 INFO:teuthology.orchestra.run.smithi181.stderr:145500K .......... .......... .......... .......... .......... 86% 24.5M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145550K .......... .......... .......... .......... .......... 86% 39.7M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145600K .......... .......... .......... .......... .......... 86% 24.6M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145650K .......... .......... .......... .......... .......... 86% 42.1M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145700K .......... .......... .......... .......... .......... 86% 24.7M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145750K .......... .......... .......... .......... .......... 86% 24.1M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145800K .......... .......... .......... .......... .......... 86% 36.5M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145850K .......... .......... .......... .......... .......... 86% 24.9M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145900K .......... .......... .......... .......... .......... 86% 46.7M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:145950K .......... .......... .......... .......... .......... 86% 24.6M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146000K .......... .......... .......... .......... .......... 86% 34.7M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146050K .......... .......... .......... .......... .......... 86% 28.9M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146100K .......... .......... .......... .......... .......... 86% 22.4M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146150K .......... .......... .......... .......... .......... 86% 48.3M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146200K .......... .......... .......... .......... .......... 86% 24.2M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146250K .......... .......... .......... .......... .......... 86% 31.8M 1s 2024-03-29T04:10:43.986 INFO:teuthology.orchestra.run.smithi181.stderr:146300K .......... .......... .......... .......... .......... 86% 24.6M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146350K .......... .......... .......... .......... .......... 86% 28.1M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146400K .......... .......... .......... .......... .......... 86% 47.7M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146450K .......... .......... .......... .......... .......... 86% 22.2M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146500K .......... .......... .......... .......... .......... 86% 31.3M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146550K .......... .......... .......... .......... .......... 86% 28.9M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146600K .......... .......... .......... .......... .......... 86% 26.0M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146650K .......... .......... .......... .......... .......... 87% 26.4M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146700K .......... .......... .......... .......... .......... 87% 38.6M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146750K .......... .......... .......... .......... .......... 87% 22.6M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146800K .......... .......... .......... .......... .......... 87% 34.7M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146850K .......... .......... .......... .......... .......... 87% 32.6M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146900K .......... .......... .......... .......... .......... 87% 22.6M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:146950K .......... .......... .......... .......... .......... 87% 40.7M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:147000K .......... .......... .......... .......... .......... 87% 24.1M 1s 2024-03-29T04:10:43.987 INFO:teuthology.orchestra.run.smithi181.stderr:147050K .......... .......... .......... .......... .......... 87% 34.6M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147100K .......... .......... .......... .......... .......... 87% 28.0M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147150K .......... .......... .......... .......... .......... 87% 23.8M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147200K .......... .......... .......... .......... .......... 87% 48.4M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147250K .......... .......... .......... .......... .......... 87% 25.5M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147300K .......... .......... .......... .......... .......... 87% 23.2M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147350K .......... .......... .......... .......... .......... 87% 37.4M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147400K .......... .......... .......... .......... .......... 87% 24.9M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147450K .......... .......... .......... .......... .......... 87% 43.8M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147500K .......... .......... .......... .......... .......... 87% 23.5M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147550K .......... .......... .......... .......... .......... 87% 35.3M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147600K .......... .......... .......... .......... .......... 87% 24.8M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147650K .......... .......... .......... .......... .......... 87% 24.4M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147700K .......... .......... .......... .......... .......... 87% 47.3M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147750K .......... .......... .......... .......... .......... 87% 24.7M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147800K .......... .......... .......... .......... .......... 87% 35.6M 1s 2024-03-29T04:10:43.988 INFO:teuthology.orchestra.run.smithi181.stderr:147850K .......... .......... .......... .......... .......... 87% 24.1M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:147900K .......... .......... .......... .......... .......... 87% 24.9M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:147950K .......... .......... .......... .......... .......... 87% 47.0M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148000K .......... .......... .......... .......... .......... 87% 24.7M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148050K .......... .......... .......... .......... .......... 87% 24.7M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148100K .......... .......... .......... .......... .......... 87% 33.4M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148150K .......... .......... .......... .......... .......... 87% 25.0M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148200K .......... .......... .......... .......... .......... 87% 49.8M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148250K .......... .......... .......... .......... .......... 87% 25.3M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148300K .......... .......... .......... .......... .......... 88% 23.9M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148350K .......... .......... .......... .......... .......... 88% 34.7M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148400K .......... .......... .......... .......... .......... 88% 24.9M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148450K .......... .......... .......... .......... .......... 88% 24.7M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148500K .......... .......... .......... .......... .......... 88% 51.6M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148550K .......... .......... .......... .......... .......... 88% 31.6M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148600K .......... .......... .......... .......... .......... 88% 26.0M 1s 2024-03-29T04:10:43.989 INFO:teuthology.orchestra.run.smithi181.stderr:148650K .......... .......... .......... .......... .......... 88% 25.0M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148700K .......... .......... .......... .......... .......... 88% 24.0M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148750K .......... .......... .......... .......... .......... 88% 51.5M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148800K .......... .......... .......... .......... .......... 88% 32.1M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148850K .......... .......... .......... .......... .......... 88% 25.9M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148900K .......... .......... .......... .......... .......... 88% 24.0M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:148950K .......... .......... .......... .......... .......... 88% 24.9M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149000K .......... .......... .......... .......... .......... 88% 50.5M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149050K .......... .......... .......... .......... .......... 88% 27.1M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149100K .......... .......... .......... .......... .......... 88% 26.2M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149150K .......... .......... .......... .......... .......... 88% 27.4M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149200K .......... .......... .......... .......... .......... 88% 24.4M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149250K .......... .......... .......... .......... .......... 88% 58.1M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149300K .......... .......... .......... .......... .......... 88% 29.8M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149350K .......... .......... .......... .......... .......... 88% 26.2M 1s 2024-03-29T04:10:43.990 INFO:teuthology.orchestra.run.smithi181.stderr:149400K .......... .......... .......... .......... .......... 88% 24.8M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149450K .......... .......... .......... .......... .......... 88% 39.1M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149500K .......... .......... .......... .......... .......... 88% 28.6M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149550K .......... .......... .......... .......... .......... 88% 27.5M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149600K .......... .......... .......... .......... .......... 88% 28.9M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149650K .......... .......... .......... .......... .......... 88% 23.8M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149700K .......... .......... .......... .......... .......... 88% 24.8M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149750K .......... .......... .......... .......... .......... 88% 50.3M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149800K .......... .......... .......... .......... .......... 88% 25.6M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149850K .......... .......... .......... .......... .......... 88% 32.8M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149900K .......... .......... .......... .......... .......... 88% 24.4M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:149950K .......... .......... .......... .......... .......... 88% 24.8M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:150000K .......... .......... .......... .......... .......... 89% 53.3M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:150050K .......... .......... .......... .......... .......... 89% 30.5M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:150100K .......... .......... .......... .......... .......... 89% 24.5M 1s 2024-03-29T04:10:43.991 INFO:teuthology.orchestra.run.smithi181.stderr:150150K .......... .......... .......... .......... .......... 89% 26.4M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150200K .......... .......... .......... .......... .......... 89% 25.1M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150250K .......... .......... .......... .......... .......... 89% 48.7M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150300K .......... .......... .......... .......... .......... 89% 24.6M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150350K .......... .......... .......... .......... .......... 89% 31.2M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150400K .......... .......... .......... .......... .......... 89% 26.3M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150450K .......... .......... .......... .......... .......... 89% 24.4M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150500K .......... .......... .......... .......... .......... 89% 53.5M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150550K .......... .......... .......... .......... .......... 89% 29.1M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150600K .......... .......... .......... .......... .......... 89% 23.8M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150650K .......... .......... .......... .......... .......... 89% 28.7M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150700K .......... .......... .......... .......... .......... 89% 23.4M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150750K .......... .......... .......... .......... .......... 89% 55.6M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150800K .......... .......... .......... .......... .......... 89% 22.9M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150850K .......... .......... .......... .......... .......... 89% 31.5M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150900K .......... .......... .......... .......... .......... 89% 27.3M 1s 2024-03-29T04:10:43.992 INFO:teuthology.orchestra.run.smithi181.stderr:150950K .......... .......... .......... .......... .......... 89% 22.2M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151000K .......... .......... .......... .......... .......... 89% 54.1M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151050K .......... .......... .......... .......... .......... 89% 24.0M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151100K .......... .......... .......... .......... .......... 89% 32.6M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151150K .......... .......... .......... .......... .......... 89% 25.7M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151200K .......... .......... .......... .......... .......... 89% 24.4M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151250K .......... .......... .......... .......... .......... 89% 24.9M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151300K .......... .......... .......... .......... .......... 89% 30.9M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151350K .......... .......... .......... .......... .......... 89% 53.4M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151400K .......... .......... .......... .......... .......... 89% 24.4M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151450K .......... .......... .......... .......... .......... 89% 26.1M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151500K .......... .......... .......... .......... .......... 89% 29.6M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151550K .......... .......... .......... .......... .......... 89% 23.9M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151600K .......... .......... .......... .......... .......... 89% 63.5M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151650K .......... .......... .......... .......... .......... 89% 24.3M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151700K .......... .......... .......... .......... .......... 90% 29.0M 1s 2024-03-29T04:10:43.993 INFO:teuthology.orchestra.run.smithi181.stderr:151750K .......... .......... .......... .......... .......... 90% 26.1M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:151800K .......... .......... .......... .......... .......... 90% 24.4M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:151850K .......... .......... .......... .......... .......... 90% 59.9M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:151900K .......... .......... .......... .......... .......... 90% 24.4M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:151950K .......... .......... .......... .......... .......... 90% 30.5M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152000K .......... .......... .......... .......... .......... 90% 24.6M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152050K .......... .......... .......... .......... .......... 90% 36.1M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152100K .......... .......... .......... .......... .......... 90% 34.0M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152150K .......... .......... .......... .......... .......... 90% 22.9M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152200K .......... .......... .......... .......... .......... 90% 31.9M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152250K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152300K .......... .......... .......... .......... .......... 90% 25.0M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152350K .......... .......... .......... .......... .......... 90% 63.2M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152400K .......... .......... .......... .......... .......... 90% 24.4M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152450K .......... .......... .......... .......... .......... 90% 30.7M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152500K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-03-29T04:10:43.994 INFO:teuthology.orchestra.run.smithi181.stderr:152550K .......... .......... .......... .......... .......... 90% 24.1M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152600K .......... .......... .......... .......... .......... 90% 26.7M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152650K .......... .......... .......... .......... .......... 90% 37.9M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152700K .......... .......... .......... .......... .......... 90% 40.4M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152750K .......... .......... .......... .......... .......... 90% 24.1M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152800K .......... .......... .......... .......... .......... 90% 25.0M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152850K .......... .......... .......... .......... .......... 90% 40.4M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152900K .......... .......... .......... .......... .......... 90% 24.6M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:152950K .......... .......... .......... .......... .......... 90% 40.3M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153000K .......... .......... .......... .......... .......... 90% 24.5M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153050K .......... .......... .......... .......... .......... 90% 24.8M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153100K .......... .......... .......... .......... .......... 90% 37.1M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153150K .......... .......... .......... .......... .......... 90% 25.6M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153200K .......... .......... .......... .......... .......... 90% 42.4M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153250K .......... .......... .......... .......... .......... 90% 24.7M 1s 2024-03-29T04:10:43.995 INFO:teuthology.orchestra.run.smithi181.stderr:153300K .......... .......... .......... .......... .......... 90% 24.3M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153350K .......... .......... .......... .......... .......... 91% 36.4M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153400K .......... .......... .......... .......... .......... 91% 25.5M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153450K .......... .......... .......... .......... .......... 91% 43.1M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153500K .......... .......... .......... .......... .......... 91% 24.7M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153550K .......... .......... .......... .......... .......... 91% 24.6M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153600K .......... .......... .......... .......... .......... 91% 37.1M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153650K .......... .......... .......... .......... .......... 91% 24.5M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153700K .......... .......... .......... .......... .......... 91% 44.0M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153750K .......... .......... .......... .......... .......... 91% 24.6M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153800K .......... .......... .......... .......... .......... 91% 24.2M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153850K .......... .......... .......... .......... .......... 91% 38.0M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153900K .......... .......... .......... .......... .......... 91% 32.9M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:153950K .......... .......... .......... .......... .......... 91% 24.8M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:154000K .......... .......... .......... .......... .......... 91% 34.6M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:154050K .......... .......... .......... .......... .......... 91% 24.8M 1s 2024-03-29T04:10:43.996 INFO:teuthology.orchestra.run.smithi181.stderr:154100K .......... .......... .......... .......... .......... 91% 23.0M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154150K .......... .......... .......... .......... .......... 91% 50.7M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154200K .......... .......... .......... .......... .......... 91% 25.0M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154250K .......... .......... .......... .......... .......... 91% 24.4M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154300K .......... .......... .......... .......... .......... 91% 29.7M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154350K .......... .......... .......... .......... .......... 91% 25.9M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154400K .......... .......... .......... .......... .......... 91% 56.4M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154450K .......... .......... .......... .......... .......... 91% 24.4M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154500K .......... .......... .......... .......... .......... 91% 28.7M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154550K .......... .......... .......... .......... .......... 91% 26.1M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154600K .......... .......... .......... .......... .......... 91% 24.1M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154650K .......... .......... .......... .......... .......... 91% 61.3M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154700K .......... .......... .......... .......... .......... 91% 26.0M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154750K .......... .......... .......... .......... .......... 91% 22.8M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154800K .......... .......... .......... .......... .......... 91% 29.4M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154850K .......... .......... .......... .......... .......... 91% 53.0M 1s 2024-03-29T04:10:43.997 INFO:teuthology.orchestra.run.smithi181.stderr:154900K .......... .......... .......... .......... .......... 91% 29.2M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:154950K .......... .......... .......... .......... .......... 91% 24.1M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155000K .......... .......... .......... .......... .......... 91% 27.3M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155050K .......... .......... .......... .......... .......... 92% 27.3M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155100K .......... .......... .......... .......... .......... 92% 24.3M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155150K .......... .......... .......... .......... .......... 92% 59.3M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155200K .......... .......... .......... .......... .......... 92% 24.8M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155250K .......... .......... .......... .......... .......... 92% 32.8M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155300K .......... .......... .......... .......... .......... 92% 20.9M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155350K .......... .......... .......... .......... .......... 92% 44.0M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155400K .......... .......... .......... .......... .......... 92% 34.8M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155450K .......... .......... .......... .......... .......... 92% 19.9M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155500K .......... .......... .......... .......... .......... 92% 27.7M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155550K .......... .......... .......... .......... .......... 92% 57.1M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155600K .......... .......... .......... .......... .......... 92% 22.4M 1s 2024-03-29T04:10:43.998 INFO:teuthology.orchestra.run.smithi181.stderr:155650K .......... .......... .......... .......... .......... 92% 35.8M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155700K .......... .......... .......... .......... .......... 92% 20.9M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155750K .......... .......... .......... .......... .......... 92% 49.2M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155800K .......... .......... .......... .......... .......... 92% 31.8M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155850K .......... .......... .......... .......... .......... 92% 25.1M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155900K .......... .......... .......... .......... .......... 92% 25.1M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:155950K .......... .......... .......... .......... .......... 92% 24.3M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156000K .......... .......... .......... .......... .......... 92% 52.5M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156050K .......... .......... .......... .......... .......... 92% 31.0M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156100K .......... .......... .......... .......... .......... 92% 25.0M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156150K .......... .......... .......... .......... .......... 92% 21.5M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156200K .......... .......... .......... .......... .......... 92% 29.3M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156250K .......... .......... .......... .......... .......... 92% 49.4M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156300K .......... .......... .......... .......... .......... 92% 34.1M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156350K .......... .......... .......... .......... .......... 92% 24.8M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156400K .......... .......... .......... .......... .......... 92% 23.6M 1s 2024-03-29T04:10:43.999 INFO:teuthology.orchestra.run.smithi181.stderr:156450K .......... .......... .......... .......... .......... 92% 24.2M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156500K .......... .......... .......... .......... .......... 92% 33.3M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156550K .......... .......... .......... .......... .......... 92% 49.5M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156600K .......... .......... .......... .......... .......... 92% 24.0M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156650K .......... .......... .......... .......... .......... 92% 37.1M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156700K .......... .......... .......... .......... .......... 92% 25.0M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156750K .......... .......... .......... .......... .......... 93% 24.5M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156800K .......... .......... .......... .......... .......... 93% 48.0M 1s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156850K .......... .......... .......... .......... .......... 93% 29.6M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156900K .......... .......... .......... .......... .......... 93% 23.2M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:156950K .......... .......... .......... .......... .......... 93% 27.4M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157000K .......... .......... .......... .......... .......... 93% 31.4M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157050K .......... .......... .......... .......... .......... 93% 24.6M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157100K .......... .......... .......... .......... .......... 93% 53.7M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157150K .......... .......... .......... .......... .......... 93% 23.7M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157200K .......... .......... .......... .......... .......... 93% 23.4M 0s 2024-03-29T04:10:44.000 INFO:teuthology.orchestra.run.smithi181.stderr:157250K .......... .......... .......... .......... .......... 93% 35.9M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157300K .......... .......... .......... .......... .......... 93% 22.8M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157350K .......... .......... .......... .......... .......... 93% 63.0M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157400K .......... .......... .......... .......... .......... 93% 23.0M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157450K .......... .......... .......... .......... .......... 93% 35.8M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157500K .......... .......... .......... .......... .......... 93% 10.2M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157550K .......... .......... .......... .......... .......... 93% 51.4M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157600K .......... .......... .......... .......... .......... 93% 590M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157650K .......... .......... .......... .......... .......... 93% 43.7M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157700K .......... .......... .......... .......... .......... 93% 26.0M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157750K .......... .......... .......... .......... .......... 93% 33.2M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157800K .......... .......... .......... .......... .......... 93% 24.7M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157850K .......... .......... .......... .......... .......... 93% 45.6M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157900K .......... .......... .......... .......... .......... 93% 24.3M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:157950K .......... .......... .......... .......... .......... 93% 32.6M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:158000K .......... .......... .......... .......... .......... 93% 23.1M 0s 2024-03-29T04:10:44.001 INFO:teuthology.orchestra.run.smithi181.stderr:158050K .......... .......... .......... .......... .......... 93% 28.2M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158100K .......... .......... .......... .......... .......... 93% 47.9M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158150K .......... .......... .......... .......... .......... 93% 25.1M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158200K .......... .......... .......... .......... .......... 93% 25.1M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158250K .......... .......... .......... .......... .......... 93% 25.5M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158300K .......... .......... .......... .......... .......... 93% 33.3M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158350K .......... .......... .......... .......... .......... 93% 23.9M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158400K .......... .......... .......... .......... .......... 93% 49.3M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158450K .......... .......... .......... .......... .......... 94% 34.5M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158500K .......... .......... .......... .......... .......... 94% 24.5M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158550K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158600K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158650K .......... .......... .......... .......... .......... 94% 52.2M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158700K .......... .......... .......... .......... .......... 94% 34.0M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158750K .......... .......... .......... .......... .......... 94% 24.3M 0s 2024-03-29T04:10:44.002 INFO:teuthology.orchestra.run.smithi181.stderr:158800K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:158850K .......... .......... .......... .......... .......... 94% 24.5M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:158900K .......... .......... .......... .......... .......... 94% 51.7M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:158950K .......... .......... .......... .......... .......... 94% 33.1M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159000K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159050K .......... .......... .......... .......... .......... 94% 24.8M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159100K .......... .......... .......... .......... .......... 94% 27.7M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159150K .......... .......... .......... .......... .......... 94% 37.7M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159200K .......... .......... .......... .......... .......... 94% 23.6M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159250K .......... .......... .......... .......... .......... 94% 40.6M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159300K .......... .......... .......... .......... .......... 94% 25.8M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159350K .......... .......... .......... .......... .......... 94% 24.4M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159400K .......... .......... .......... .......... .......... 94% 37.8M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159450K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159500K .......... .......... .......... .......... .......... 94% 35.3M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159550K .......... .......... .......... .......... .......... 94% 20.0M 0s 2024-03-29T04:10:44.003 INFO:teuthology.orchestra.run.smithi181.stderr:159600K .......... .......... .......... .......... .......... 94% 31.9M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159650K .......... .......... .......... .......... .......... 94% 31.9M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159700K .......... .......... .......... .......... .......... 94% 41.2M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159750K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159800K .......... .......... .......... .......... .......... 94% 38.3M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159850K .......... .......... .......... .......... .......... 94% 23.4M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159900K .......... .......... .......... .......... .......... 94% 24.5M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:159950K .......... .......... .......... .......... .......... 94% 50.5M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160000K .......... .......... .......... .......... .......... 94% 23.7M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160050K .......... .......... .......... .......... .......... 94% 36.9M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160100K .......... .......... .......... .......... .......... 95% 24.6M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160150K .......... .......... .......... .......... .......... 95% 24.7M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160200K .......... .......... .......... .......... .......... 95% 44.7M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160250K .......... .......... .......... .......... .......... 95% 25.0M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160300K .......... .......... .......... .......... .......... 95% 36.3M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160350K .......... .......... .......... .......... .......... 95% 24.4M 0s 2024-03-29T04:10:44.004 INFO:teuthology.orchestra.run.smithi181.stderr:160400K .......... .......... .......... .......... .......... 95% 24.2M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160450K .......... .......... .......... .......... .......... 95% 47.5M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160500K .......... .......... .......... .......... .......... 95% 24.7M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160550K .......... .......... .......... .......... .......... 95% 37.0M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160600K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160650K .......... .......... .......... .......... .......... 95% 24.4M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160700K .......... .......... .......... .......... .......... 95% 46.6M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160750K .......... .......... .......... .......... .......... 95% 24.7M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160800K .......... .......... .......... .......... .......... 95% 35.3M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160850K .......... .......... .......... .......... .......... 95% 24.9M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160900K .......... .......... .......... .......... .......... 95% 25.0M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:160950K .......... .......... .......... .......... .......... 95% 42.9M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:161000K .......... .......... .......... .......... .......... 95% 19.5M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:161050K .......... .......... .......... .......... .......... 95% 51.7M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:161100K .......... .......... .......... .......... .......... 95% 25.4M 0s 2024-03-29T04:10:44.005 INFO:teuthology.orchestra.run.smithi181.stderr:161150K .......... .......... .......... .......... .......... 95% 32.8M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161200K .......... .......... .......... .......... .......... 95% 24.5M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161250K .......... .......... .......... .......... .......... 95% 24.7M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161300K .......... .......... .......... .......... .......... 95% 54.4M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161350K .......... .......... .......... .......... .......... 95% 24.3M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161400K .......... .......... .......... .......... .......... 95% 31.1M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161450K .......... .......... .......... .......... .......... 95% 24.6M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161500K .......... .......... .......... .......... .......... 95% 25.5M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161550K .......... .......... .......... .......... .......... 95% 50.8M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161600K .......... .......... .......... .......... .......... 95% 24.8M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161650K .......... .......... .......... .......... .......... 95% 31.8M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161700K .......... .......... .......... .......... .......... 95% 24.8M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161750K .......... .......... .......... .......... .......... 95% 29.0M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161800K .......... .......... .......... .......... .......... 96% 39.1M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161850K .......... .......... .......... .......... .......... 96% 24.0M 0s 2024-03-29T04:10:44.006 INFO:teuthology.orchestra.run.smithi181.stderr:161900K .......... .......... .......... .......... .......... 96% 23.6M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:161950K .......... .......... .......... .......... .......... 96% 49.0M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162000K .......... .......... .......... .......... .......... 96% 24.5M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162050K .......... .......... .......... .......... .......... 96% 36.1M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162100K .......... .......... .......... .......... .......... 96% 25.3M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162150K .......... .......... .......... .......... .......... 96% 34.8M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162200K .......... .......... .......... .......... .......... 96% 20.4M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162250K .......... .......... .......... .......... .......... 96% 31.6M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162300K .......... .......... .......... .......... .......... 96% 51.8M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162350K .......... .......... .......... .......... .......... 96% 27.4M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162400K .......... .......... .......... .......... .......... 96% 20.0M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162450K .......... .......... .......... .......... .......... 96% 56.2M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162500K .......... .......... .......... .......... .......... 96% 20.3M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162550K .......... .......... .......... .......... .......... 96% 49.1M 0s 2024-03-29T04:10:44.007 INFO:teuthology.orchestra.run.smithi181.stderr:162600K .......... .......... .......... .......... .......... 96% 30.2M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162650K .......... .......... .......... .......... .......... 96% 27.1M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162700K .......... .......... .......... .......... .......... 96% 24.7M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162750K .......... .......... .......... .......... .......... 96% 24.0M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162800K .......... .......... .......... .......... .......... 96% 53.1M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162850K .......... .......... .......... .......... .......... 96% 29.7M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162900K .......... .......... .......... .......... .......... 96% 23.1M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:162950K .......... .......... .......... .......... .......... 96% 30.1M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163000K .......... .......... .......... .......... .......... 96% 24.5M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163050K .......... .......... .......... .......... .......... 96% 50.2M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163100K .......... .......... .......... .......... .......... 96% 29.9M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163150K .......... .......... .......... .......... .......... 96% 26.0M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163200K .......... .......... .......... .......... .......... 96% 25.7M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163250K .......... .......... .......... .......... .......... 96% 23.9M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163300K .......... .......... .......... .......... .......... 96% 53.6M 0s 2024-03-29T04:10:44.008 INFO:teuthology.orchestra.run.smithi181.stderr:163350K .......... .......... .......... .......... .......... 96% 29.1M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163400K .......... .......... .......... .......... .......... 96% 25.7M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163450K .......... .......... .......... .......... .......... 96% 25.1M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163500K .......... .......... .......... .......... .......... 97% 24.9M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163550K .......... .......... .......... .......... .......... 97% 56.1M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163600K .......... .......... .......... .......... .......... 97% 28.8M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163650K .......... .......... .......... .......... .......... 97% 19.6M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163700K .......... .......... .......... .......... .......... 97% 41.9M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163750K .......... .......... .......... .......... .......... 97% 22.9M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163800K .......... .......... .......... .......... .......... 97% 49.3M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163850K .......... .......... .......... .......... .......... 97% 22.4M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163900K .......... .......... .......... .......... .......... 97% 24.5M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:163950K .......... .......... .......... .......... .......... 97% 53.8M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:164000K .......... .......... .......... .......... .......... 97% 29.8M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:164050K .......... .......... .......... .......... .......... 97% 26.1M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:164100K .......... .......... .......... .......... .......... 97% 25.1M 0s 2024-03-29T04:10:44.009 INFO:teuthology.orchestra.run.smithi181.stderr:164150K .......... .......... .......... .......... .......... 97% 24.2M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164200K .......... .......... .......... .......... .......... 97% 49.4M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164250K .......... .......... .......... .......... .......... 97% 32.4M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164300K .......... .......... .......... .......... .......... 97% 26.0M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164350K .......... .......... .......... .......... .......... 97% 24.4M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164400K .......... .......... .......... .......... .......... 97% 27.6M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164450K .......... .......... .......... .......... .......... 97% 34.2M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164500K .......... .......... .......... .......... .......... 97% 34.1M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164550K .......... .......... .......... .......... .......... 97% 25.6M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164600K .......... .......... .......... .......... .......... 97% 33.2M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164650K .......... .......... .......... .......... .......... 97% 24.9M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164700K .......... .......... .......... .......... .......... 97% 35.0M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164750K .......... .......... .......... .......... .......... 97% 32.7M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164800K .......... .......... .......... .......... .......... 97% 25.2M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164850K .......... .......... .......... .......... .......... 97% 27.5M 0s 2024-03-29T04:10:44.010 INFO:teuthology.orchestra.run.smithi181.stderr:164900K .......... .......... .......... .......... .......... 97% 20.1M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:164950K .......... .......... .......... .......... .......... 97% 48.3M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165000K .......... .......... .......... .......... .......... 97% 34.2M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165050K .......... .......... .......... .......... .......... 97% 24.3M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165100K .......... .......... .......... .......... .......... 97% 32.6M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165150K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165200K .......... .......... .......... .......... .......... 98% 35.0M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165250K .......... .......... .......... .......... .......... 98% 22.8M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165300K .......... .......... .......... .......... .......... 98% 34.8M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165350K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165400K .......... .......... .......... .......... .......... 98% 24.6M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165450K .......... .......... .......... .......... .......... 98% 48.4M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165500K .......... .......... .......... .......... .......... 98% 24.5M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165550K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165600K .......... .......... .......... .......... .......... 98% 32.9M 0s 2024-03-29T04:10:44.011 INFO:teuthology.orchestra.run.smithi181.stderr:165650K .......... .......... .......... .......... .......... 98% 48.4M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165700K .......... .......... .......... .......... .......... 98% 25.5M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165750K .......... .......... .......... .......... .......... 98% 24.2M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165800K .......... .......... .......... .......... .......... 98% 33.0M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165850K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165900K .......... .......... .......... .......... .......... 98% 59.7M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:165950K .......... .......... .......... .......... .......... 98% 24.3M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166000K .......... .......... .......... .......... .......... 98% 24.5M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166050K .......... .......... .......... .......... .......... 98% 30.5M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166100K .......... .......... .......... .......... .......... 98% 24.1M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166150K .......... .......... .......... .......... .......... 98% 55.0M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166200K .......... .......... .......... .......... .......... 98% 25.2M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166250K .......... .......... .......... .......... .......... 98% 23.4M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166300K .......... .......... .......... .......... .......... 98% 33.6M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166350K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166400K .......... .......... .......... .......... .......... 98% 45.0M 0s 2024-03-29T04:10:44.012 INFO:teuthology.orchestra.run.smithi181.stderr:166450K .......... .......... .......... .......... .......... 98% 26.3M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166500K .......... .......... .......... .......... .......... 98% 23.9M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166550K .......... .......... .......... .......... .......... 98% 36.1M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166600K .......... .......... .......... .......... .......... 98% 33.0M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166650K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166700K .......... .......... .......... .......... .......... 98% 24.0M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166750K .......... .......... .......... .......... .......... 98% 25.6M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166800K .......... .......... .......... .......... .......... 98% 47.0M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166850K .......... .......... .......... .......... .......... 99% 33.2M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166900K .......... .......... .......... .......... .......... 99% 25.1M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:166950K .......... .......... .......... .......... .......... 99% 23.5M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167000K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167050K .......... .......... .......... .......... .......... 99% 48.2M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167100K .......... .......... .......... .......... .......... 99% 33.5M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167150K .......... .......... .......... .......... .......... 99% 23.0M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167200K .......... .......... .......... .......... .......... 99% 31.7M 0s 2024-03-29T04:10:44.013 INFO:teuthology.orchestra.run.smithi181.stderr:167250K .......... .......... .......... .......... .......... 99% 27.5M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167300K .......... .......... .......... .......... .......... 99% 24.6M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167350K .......... .......... .......... .......... .......... 99% 50.2M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167400K .......... .......... .......... .......... .......... 99% 25.5M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167450K .......... .......... .......... .......... .......... 99% 24.3M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167500K .......... .......... .......... .......... .......... 99% 30.4M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167550K .......... .......... .......... .......... .......... 99% 29.5M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167600K .......... .......... .......... .......... .......... 99% 26.8M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167650K .......... .......... .......... .......... .......... 99% 36.5M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167700K .......... .......... .......... .......... .......... 99% 23.2M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167750K .......... .......... .......... .......... .......... 99% 34.6M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167800K .......... .......... .......... .......... .......... 99% 23.9M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167850K .......... .......... .......... .......... .......... 99% 30.2M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167900K .......... .......... .......... .......... .......... 99% 33.1M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:167950K .......... .......... .......... .......... .......... 99% 25.6M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:168000K .......... .......... .......... .......... .......... 99% 56.1M 0s 2024-03-29T04:10:44.014 INFO:teuthology.orchestra.run.smithi181.stderr:168050K .......... .......... .......... .......... .......... 99% 24.7M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168100K .......... .......... .......... .......... .......... 99% 24.5M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168150K .......... .......... .......... .......... .......... 99% 34.1M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168200K .......... .......... .......... .......... .......... 99% 25.8M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168250K .......... .......... .......... .......... .......... 99% 43.7M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168300K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168350K .......... .......... .......... .......... .......... 99% 30.5M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168400K .......... .......... .......... .......... .......... 99% 24.0M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168450K .......... .......... .......... .......... .......... 99% 24.6M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168500K .......... .......... .......... .......... .......... 99% 51.8M 0s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:168550K .......... .......... . 100% 38.7M=7.1s 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29 04:10:42 (23.2 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-03-29T04:10:44.015 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:10:44.017 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-29T04:10:44.017 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-29T04:10:44.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-29T04:10:44.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.143+0000 7f8cd46eb640 1 Processor -- start 2024-03-29T04:10:44.123 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.143+0000 7f8cd46eb640 1 -- start start 2024-03-29T04:10:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd46eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 0x7f8ccc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.124 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd46eb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0ba090 0x7f8ccc0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd46eb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ccc0b66e0 con 0x7f8ccc06c620 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd46eb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8ccc0d2ff0 con 0x7f8ccc0ba090 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd46eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc0d32d0 con 0x7f8ccc0b5dc0 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0ba090 0x7f8ccc0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0ba090 0x7f8ccc0d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42570/0 (socket says 172.21.15.181:42570) 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 -- 172.21.15.181:0/3561784028 learned_addr learned my addr 172.21.15.181:0/3561784028 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd2487640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 0x7f8ccc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 -- 172.21.15.181:0/3561784028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 msgr2=0x7f8ccc0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 0x7f8ccc0b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 -- 172.21.15.181:0/3561784028 >> v1:172.21.15.187:6789/0 conn(0x7f8ccc06c620 legacy=0x7f8ccc06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd1c86640 1 -- 172.21.15.181:0/3561784028 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc0d35b0 con 0x7f8ccc0ba090 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.144+0000 7f8cd2487640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 0x7f8ccc0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.145+0000 7f8cd1c86640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc0ba090 0x7f8ccc0d2680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009870 tx=0x7f8cbc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2137c780a5371204 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.145+0000 7f8cd1485640 1 -- 172.21.15.181:0/3561784028 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc035020 con 0x7f8ccc0ba090 2024-03-29T04:10:44.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.145+0000 7f8cd1485640 1 -- 172.21.15.181:0/3561784028 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cbc031640 con 0x7f8ccc0ba090 2024-03-29T04:10:44.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.145+0000 7f8cd1485640 1 -- 172.21.15.181:0/3561784028 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc02d040 con 0x7f8ccc0ba090 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 -- 172.21.15.181:0/3561784028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc0ba090 msgr2=0x7f8ccc0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc0ba090 0x7f8ccc0d2680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009870 tx=0x7f8cbc02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 -- 172.21.15.181:0/3561784028 shutdown_connections 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc0ba090 0x7f8ccc0d2680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/3561784028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0b5dc0 0x7f8ccc0b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 -- 172.21.15.181:0/3561784028 >> 172.21.15.181:0/3561784028 conn(0x7f8ccc0678f0 msgr2=0x7f8ccc0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 -- 172.21.15.181:0/3561784028 shutdown_connections 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.146+0000 7f8cd46eb640 1 -- 172.21.15.181:0/3561784028 wait complete. 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.147+0000 7f8cd46eb640 1 Processor -- start 2024-03-29T04:10:44.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.147+0000 7f8cd46eb640 1 -- start start 2024-03-29T04:10:44.129 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ccc0b5dc0 0x7f8ccc160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0ba090 0x7f8ccc15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0d4090 con 0x7f8ccc06c620 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0d3b70 con 0x7f8ccc0b5dc0 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd46eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc0b83e0 con 0x7f8ccc0ba090 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:42576/0 (socket says 172.21.15.181:42576) 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 -- 172.21.15.181:0/1668221130 learned_addr learned my addr 172.21.15.181:0/1668221130 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd1c86640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ccc0b5dc0 0x7f8ccc160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2c88640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0ba090 0x7f8ccc15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 -- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0ba090 msgr2=0x7f8ccc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0ba090 0x7f8ccc15c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.130 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 -- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ccc0b5dc0 msgr2=0x7f8ccc160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ccc0b5dc0 0x7f8ccc160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 -- 172.21.15.181:0/1668221130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc160990 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cd2487640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800c7f0 tx=0x7f8cc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.148+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8015070 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.149+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc8007a30 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.149+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc160c70 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.149+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8007e00 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.149+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc161230 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.150+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc075890 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.150+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8cc8018070 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.150+0000 7f8cc2ffd640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 0x7f8cb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.150+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f8cc808b870 con 0x7f8ccc06c620 2024-03-29T04:10:44.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.150+0000 7f8cd1c86640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 0x7f8cb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:44.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.151+0000 7f8cd1c86640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 0x7f8cb006e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc036000 tx=0x7f8cbc03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:44.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.155+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8cc8059d00 con 0x7f8ccc06c620 2024-03-29T04:10:44.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:44.374+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f8ccc0b8860 con 0x7f8ccc06c620 2024-03-29T04:10:45.333 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.354+0000 7f8cc2ffd640 1 -- 172.21.15.181:0/1668221130 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f8cc8059600 con 0x7f8ccc06c620 2024-03-29T04:10:45.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 msgr2=0x7f8cb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 0x7f8cb006e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc036000 tx=0x7f8cbc03c040 comp rx=0 tx=0).stop 2024-03-29T04:10:45.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 msgr2=0x7f8ccc1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:45.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800c7f0 tx=0x7f8cc800cbf0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 shutdown_connections 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8cb006c530 0x7f8cb006e9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ccc0ba090 0x7f8ccc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ccc0b5dc0 0x7f8ccc160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 --2- 172.21.15.181:0/1668221130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc06c620 0x7f8ccc1518b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 >> 172.21.15.181:0/1668221130 conn(0x7f8ccc0678f0 msgr2=0x7f8ccc15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 shutdown_connections 2024-03-29T04:10:45.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:45.358+0000 7f8cd46eb640 1 -- 172.21.15.181:0/1668221130 wait complete. 2024-03-29T04:10:45.353 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-29T04:10:45.353 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-29T04:10:45.353 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-29T04:10:45.353 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-29T04:10:49.650 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-29T04:10:49.650 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-29T04:10:49.681 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph 2024-03-29T04:10:49.682 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2 2024-03-29T04:10:49.682 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs 2024-03-29T04:10:49.682 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-29T04:10:49.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:49.683 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-29T04:10:49.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:49.739 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-29T04:10:49.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:49.795 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-29T04:10:49.851 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-29T04:10:49.851 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph/Makefile 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph/io.h 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/Makefile 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:10:49.916 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/acl.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:10:49.917 INFO:teuthology.orchestra.run.smithi181.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:10:49.917 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-29T04:10:49.918 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-29T04:10:49.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:49.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-29T04:10:50.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.082+0000 7f747142a640 1 Processor -- start 2024-03-29T04:10:50.062 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.082+0000 7f747142a640 1 -- start start 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f747142a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746c06d4a0 0x7f746c0d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f747142a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c0d3ef0 0x7f746c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f747142a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c0d2910 con 0x7f746c15e440 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f747142a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0d3520 con 0x7f746c06d4a0 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f747142a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0d2bf0 con 0x7f746c0d3ef0 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746c06d4a0 0x7f746c0d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f746affd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f746c15e440 0x7f746c0b9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52652/0 (socket says 172.21.15.181:52652) 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f746affd640 1 -- 172.21.15.181:0/2543368712 learned_addr learned my addr 172.21.15.181:0/2543368712 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746c06d4a0 0x7f746c0d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39810/0 (socket says 172.21.15.181:39810) 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 -- 172.21.15.181:0/2543368712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c0d3ef0 msgr2=0x7f746c0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 --2- 172.21.15.181:0/2543368712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c0d3ef0 0x7f746c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 -- 172.21.15.181:0/2543368712 >> v1:172.21.15.181:6789/0 conn(0x7f746c15e440 legacy=0x7f746c0b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f746a7fc640 1 -- 172.21.15.181:0/2543368712 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787699479 0 0) 0x7f746c0d2910 con 0x7f746c15e440 2024-03-29T04:10:50.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.083+0000 7f74627fc640 1 -- 172.21.15.181:0/2543368712 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c0d2ed0 con 0x7f746c06d4a0 2024-03-29T04:10:50.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.084+0000 7f74627fc640 1 --2- 172.21.15.181:0/2543368712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06d4a0 0x7f746c0d39b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f74540097a0 tx=0x7f745402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3335debede8ed80a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f746a7fc640 1 -- 172.21.15.181:0/2543368712 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454034070 con 0x7f746c06d4a0 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f746a7fc640 1 -- 172.21.15.181:0/2543368712 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7454030610 con 0x7f746c06d4a0 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f746a7fc640 1 -- 172.21.15.181:0/2543368712 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540309e0 con 0x7f746c06d4a0 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 -- 172.21.15.181:0/2543368712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06d4a0 msgr2=0x7f746c0d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 --2- 172.21.15.181:0/2543368712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06d4a0 0x7f746c0d39b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f74540097a0 tx=0x7f745402b960 comp rx=0 tx=0).stop 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 -- 172.21.15.181:0/2543368712 shutdown_connections 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 --2- 172.21.15.181:0/2543368712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c0d3ef0 0x7f746c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 --2- 172.21.15.181:0/2543368712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c06d4a0 0x7f746c0d39b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 -- 172.21.15.181:0/2543368712 >> 172.21.15.181:0/2543368712 conn(0x7f746c0676b0 msgr2=0x7f746c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 -- 172.21.15.181:0/2543368712 shutdown_connections 2024-03-29T04:10:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.085+0000 7f747142a640 1 -- 172.21.15.181:0/2543368712 wait complete. 2024-03-29T04:10:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.086+0000 7f747142a640 1 Processor -- start 2024-03-29T04:10:50.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.086+0000 7f747142a640 1 -- start start 2024-03-29T04:10:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c06d4a0 0x7f746c1f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 0x7f746c0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.069 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f746c15e9a0 con 0x7f746c15e440 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f746c15e820 con 0x7f746c06d4a0 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f747142a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0cf580 con 0x7f746c154910 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f74627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f74627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39816/0 (socket says 172.21.15.181:39816) 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.087+0000 7f74627fc640 1 -- 172.21.15.181:0/3393702840 learned_addr learned my addr 172.21.15.181:0/3393702840 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 -- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 msgr2=0x7f746c0cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f746b7fe640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 0x7f746c0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f746affd640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c06d4a0 0x7f746c1f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 0x7f746c0cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 -- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c06d4a0 msgr2=0x7f746c1f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c06d4a0 0x7f746c1f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.070 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 -- 172.21.15.181:0/3393702840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c1f4cf0 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f746b7fe640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 0x7f746c0cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f74627fc640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f74540097a0 tx=0x7f74540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f7463fff640 1 -- 172.21.15.181:0/3393702840 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454034040 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f7463fff640 1 -- 172.21.15.181:0/3393702840 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7454004030 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c1f4fd0 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f7463fff640 1 -- 172.21.15.181:0/3393702840 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454004420 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.088+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c1f5590 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.089+0000 7f7463fff640 1 -- 172.21.15.181:0/3393702840 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7454037070 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.090+0000 7f7463fff640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 0x7f744c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.090+0000 7f7463fff640 1 -- 172.21.15.181:0/3393702840 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f74540ad6f0 con 0x7f746c15e440 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.090+0000 7f746affd640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 0x7f744c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.090+0000 7f746affd640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 0x7f744c06ea20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f745c000f60 tx=0x7f745c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.091+0000 7f7474011740 1 Processor -- start 2024-03-29T04:10:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.091+0000 7f7474011740 1 -- start start 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b601b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b60122410 con 0x558b601b9000 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b600c0a80 con 0x558b601e5f00 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558b6011e3c0 con 0x558b601e5b20 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39830/0 (socket says 172.21.15.181:39830) 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 -- 172.21.15.181:0/554357493 learned_addr learned my addr 172.21.15.181:0/554357493 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f74627fc640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746affd640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b601b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 -- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 msgr2=0x558b601b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b601b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 -- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 msgr2=0x558b601e88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 -- 172.21.15.181:0/554357493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b5ff027b0 con 0x558b601b9000 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746affd640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b601b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f746b7fe640 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7464007f00 tx=0x7f746400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676f815ff5ab6e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7460ff9640 1 -- 172.21.15.181:0/554357493 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464014070 con 0x558b601b9000 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7460ff9640 1 -- 172.21.15.181:0/554357493 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746400f650 con 0x558b601b9000 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.092+0000 7f7460ff9640 1 -- 172.21.15.181:0/554357493 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400fa20 con 0x558b601b9000 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 -- 172.21.15.181:0/554357493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 msgr2=0x558b601e8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7464007f00 tx=0x7f746400c690 comp rx=0 tx=0).stop 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 -- 172.21.15.181:0/554357493 shutdown_connections 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601b9000 0x558b601e8df0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 --2- 172.21.15.181:0/554357493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b601b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 -- 172.21.15.181:0/554357493 >> 172.21.15.181:0/554357493 conn(0x558b601bad30 msgr2=0x558b602038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 -- 172.21.15.181:0/554357493 shutdown_connections 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.093+0000 7f7474011740 1 -- 172.21.15.181:0/554357493 wait complete. 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.094+0000 7f7474011740 1 -- start start 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.094+0000 7f7474011740 1 Processor -- start 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b60214a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b601bd3d0 con 0x558b601df280 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b600be740 con 0x558b601e5f00 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558b601bfc10 con 0x558b601e5b20 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 10 client.0 _ll_register_callbacks cb 0x558b600cbb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:50.081 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39834/0 (socket says 172.21.15.181:39834) 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 -- 172.21.15.181:0/711824831 learned_addr learned my addr 172.21.15.181:0/711824831 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746affd640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b60214a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f74627fc640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 -- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 msgr2=0x558b60214a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b60214a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 -- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 msgr2=0x558b601ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 -- 172.21.15.181:0/711824831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b5ff62f40 con 0x558b601df280 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f74627fc640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746affd640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b60214a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f746b7fe640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7464000f80 tx=0x7f7464004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1549e3bef70e25f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.082 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464029060 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746400f650 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.095+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f7464024470 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f746402f050 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 10 client.5619 fetch_fsmap learned FSMap version 18 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558b5fdf60a0 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746400f650 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f7464032020 con 0x558b601df280 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 10 client.5619 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 10 client.5619 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 20 client.5619 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60217620 0x558b60217a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 4 client.5619 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:10:50.083 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6012f950 con 0x558b60217620 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f746affd640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60217620 0x558b60217a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f746affd640 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60217620 0x558b60217a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395c4216ca7d9dc9 server_cookie=838ca5ca7f80de84 in_seq=0 out_seq=0 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.096+0000 7f7449ffb640 10 client.5619 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7449ffb640 1 -- 172.21.15.181:0/711824831 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x558b6012f950 con 0x558b60217620 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7449ffb640 10 client.5619 handle_command_reply: tid=0 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 client.5619 shutdown 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 msgr2=0x558b601e0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7464000f80 tx=0x7f7464004a40 comp rx=0 tx=0).stop 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 shutdown_connections 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60217620 0x558b60217a30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e0d00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7449ffb640 0 client.5619 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 --2- 172.21.15.181:0/711824831 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5b20 0x558b60214a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 >> 172.21.15.181:0/711824831 conn(0x558b601b9000 msgr2=0x558b60205c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 shutdown_connections 2024-03-29T04:10:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 1 -- 172.21.15.181:0/711824831 wait complete. 2024-03-29T04:10:50.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.097+0000 7f7474011740 20 client.5619 trim_cache size 0 max 16384 2024-03-29T04:10:50.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 Processor -- start 2024-03-29T04:10:50.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 -- start start 2024-03-29T04:10:50.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5b20 0x558b601ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5f00 0x558b601ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f746c201120 con 0x558b601df280 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f746c200180 con 0x558b601e5b20 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0ca550 con 0x558b601e5f00 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f746affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f746affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39842/0 (socket says 172.21.15.181:39842) 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.115+0000 7f746affd640 1 -- 172.21.15.181:0/1089556138 learned_addr learned my addr 172.21.15.181:0/1089556138 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 -- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5f00 msgr2=0x558b601ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.097 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5f00 0x558b601ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 -- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5b20 msgr2=0x558b601ebb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5b20 0x558b601ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 -- 172.21.15.181:0/1089556138 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b60217da0 con 0x558b601df280 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f746affd640 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f745c007fd0 tx=0x7f745c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe450bb954f92713 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f744bfff640 1 -- 172.21.15.181:0/1089556138 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c02a070 con 0x558b601df280 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f744bfff640 1 -- 172.21.15.181:0/1089556138 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c005c80 con 0x558b601df280 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f744bfff640 1 -- 172.21.15.181:0/1089556138 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c00db60 con 0x558b601df280 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 -- 172.21.15.181:0/1089556138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 msgr2=0x558b601e4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f745c007fd0 tx=0x7f745c011000 comp rx=0 tx=0).stop 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 -- 172.21.15.181:0/1089556138 shutdown_connections 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601e5f00 0x558b601ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5b20 0x558b601ebb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.098 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 --2- 172.21.15.181:0/1089556138 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601df280 0x558b601e4a10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.116+0000 7f7474011740 1 -- 172.21.15.181:0/1089556138 >> 172.21.15.181:0/1089556138 conn(0x558b601b9000 msgr2=0x558b60203290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.117+0000 7f7474011740 1 -- 172.21.15.181:0/1089556138 shutdown_connections 2024-03-29T04:10:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.117+0000 7f7474011740 1 -- 172.21.15.181:0/1089556138 wait complete. 2024-03-29T04:10:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.117+0000 7f7474011740 1 -- start start 2024-03-29T04:10:50.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.117+0000 7f7474011740 1 Processor -- start 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601df280 0x558b6027f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b60205e20 con 0x558b601e5b20 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f746c1ffb30 con 0x558b601e5f00 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x558b60280050 con 0x558b601df280 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7474011740 10 client.0 _ll_register_callbacks cb 0x7f7454009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39852/0 (socket says 172.21.15.181:39852) 2024-03-29T04:10:50.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 -- 172.21.15.181:0/2463191375 learned_addr learned my addr 172.21.15.181:0/2463191375 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f746affd640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601df280 0x558b6027f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f746b7fe640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 -- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601df280 msgr2=0x558b6027f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601df280 0x558b6027f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 -- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 msgr2=0x558b601e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 -- 172.21.15.181:0/2463191375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b601ec320 con 0x558b601e5b20 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f746b7fe640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f74627fc640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7454006f80 tx=0x7f745407bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b170e94014905f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745407a9f0 con 0x558b601e5b20 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.118+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f745407ab90 con 0x558b601e5b20 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745c005c80 con 0x558b601e5b20 2024-03-29T04:10:50.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7454080ae0 con 0x558b601e5b20 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 10 client.5625 fetch_fsmap learned FSMap version 18 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x558b601ecc80 con 0x558b601e5b20 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c005c80 con 0x558b601e5b20 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f745402f050 con 0x558b601e5b20 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 10 client.5625 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 10 client.5625 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 20 client.5625 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60226cf0 0x558b602270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 4 client.5625 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x558b60129ca0 con 0x558b60226cf0 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f746affd640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60226cf0 0x558b602270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f7449ffb640 10 client.5625 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.119+0000 7f746affd640 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60226cf0 0x558b602270d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e63bdc5b2f24298 server_cookie=2b4734b0f28c7ddb in_seq=0 out_seq=0 2024-03-29T04:10:50.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.120+0000 7f7449ffb640 1 -- 172.21.15.181:0/2463191375 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b60129ca0 con 0x558b60226cf0 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.120+0000 7f7449ffb640 10 client.5625 handle_command_reply: tid=0 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.120+0000 7f7474011740 1 client.5625 shutdown 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 msgr2=0x558b601e48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7454006f80 tx=0x7f745407bde0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 shutdown_connections 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x558b60226cf0 0x558b602270d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b601e5f00 0x558b601e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7449ffb640 0 client.5625 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b601e5b20 0x558b601e48a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 --2- 172.21.15.181:0/2463191375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x558b601df280 0x558b6027f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 >> 172.21.15.181:0/2463191375 conn(0x558b601b9000 msgr2=0x558b601e02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 shutdown_connections 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 1 -- 172.21.15.181:0/2463191375 wait complete. 2024-03-29T04:10:50.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f7474011740 20 client.5625 trim_cache size 0 max 16384 2024-03-29T04:10:50.107 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:10:50.107 INFO:teuthology.orchestra.run.smithi181.stdout: "return_code": 0, 2024-03-29T04:10:50.107 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_tag": "2842fbf9-7715-41c9-99ad-42a9c182c2f2", 2024-03-29T04:10:50.107 INFO:teuthology.orchestra.run.smithi181.stdout: "mode": "asynchronous" 2024-03-29T04:10:50.107 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 msgr2=0x7f744c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 0x7f744c06ea20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f745c000f60 tx=0x7f745c009040 comp rx=0 tx=0).stop 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 msgr2=0x7f746c0cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.121+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f74540097a0 tx=0x7f74540046c0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 shutdown_connections 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f744c06c560 0x7f744c06ea20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f746c154910 0x7f746c0cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746c15e440 0x7f746c0cb670 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 --2- 172.21.15.181:0/3393702840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c06d4a0 0x7f746c1f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 >> 172.21.15.181:0/3393702840 conn(0x7f746c0676b0 msgr2=0x7f746c15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 shutdown_connections 2024-03-29T04:10:50.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.122+0000 7f747142a640 1 -- 172.21.15.181:0/3393702840 wait complete. 2024-03-29T04:10:50.116 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': '2842fbf9-7715-41c9-99ad-42a9c182c2f2', 'mode': 'asynchronous'} 2024-03-29T04:10:50.116 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-29T04:10:50.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:10:50.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:10:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.286+0000 7f5f35b90640 1 Processor -- start 2024-03-29T04:10:50.265 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.286+0000 7f5f35b90640 1 -- start start 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f35b90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f30162070 0x7f5f3015f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f35b90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f30162610 con 0x7f5f300afd60 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f2f7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5f300afd60 0x7f5f300b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:52656/0 (socket says 172.21.15.181:52656) 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/3086635932 learned_addr learned my addr 172.21.15.181:0/3086635932 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f301628f0 con 0x7f5f300b3e60 2024-03-29T04:10:50.267 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.287+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3006fac0 con 0x7f5f30162070 2024-03-29T04:10:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182097201 0 0) 0x7f5f30162610 con 0x7f5f300afd60 2024-03-29T04:10:50.270 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f18003520 con 0x7f5f300afd60 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/3086635932 >> v2:172.21.15.181:3300/0 conn(0x7f5f30162070 0x7f5f3015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 833671454 0 0) 0x7f5f18003520 con 0x7f5f300afd60 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 >> v2:172.21.15.181:3300/0 conn(0x7f5f30162070 msgr2=0x7f5f3015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 --2- 172.21.15.181:0/3086635932 >> v2:172.21.15.181:3300/0 conn(0x7f5f30162070 0x7f5f3015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 >> v1:172.21.15.187:6789/0 conn(0x7f5f300b3e60 legacy=0x7f5f3006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f30162bd0 con 0x7f5f300afd60 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f5f20001670 con 0x7f5f300afd60 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 >> v1:172.21.15.181:6789/0 conn(0x7f5f300afd60 legacy=0x7f5f300b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f18003a00 0x7f5f18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f18004800 0x7f5f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 0x7f5f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3006fac0 con 0x7f5f18003a00 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1800c090 con 0x7f5f180051c0 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1800c290 con 0x7f5f18004800 2024-03-29T04:10:50.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f18003a00 0x7f5f18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.288+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 0x7f5f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2ffff640 1 -- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f18004800 msgr2=0x7f5f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2effd640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f18004800 0x7f5f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f18004800 0x7f5f18004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2ffff640 1 -- 172.21.15.181:0/3086635932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f18003a00 msgr2=0x7f5f18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f18003a00 0x7f5f18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2ffff640 1 -- 172.21.15.181:0/3086635932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f18003520 con 0x7f5f180051c0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.289+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f18003a00 0x7f5f18003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.290+0000 7f5f2ffff640 1 -- 172.21.15.181:0/3086635932 reap_dead start 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.290+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 0x7f5f1800b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fc0 tx=0x7f5f18005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496173831448bebc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.290+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28009c10 con 0x7f5f180051c0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.290+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2802cc60 con 0x7f5f180051c0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.290+0000 7f5f2e7fc640 1 -- 172.21.15.181:0/3086635932 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28003bd0 con 0x7f5f180051c0 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 msgr2=0x7f5f1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 0x7f5f1800b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009fc0 tx=0x7f5f18005830 comp rx=0 tx=0).stop 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 shutdown_connections 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 --2- 172.21.15.181:0/3086635932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f180051c0 0x7f5f1800b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 >> 172.21.15.181:0/3086635932 conn(0x7f5f300d0840 msgr2=0x7f5f300d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 shutdown_connections 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.291+0000 7f5f35b90640 1 -- 172.21.15.181:0/3086635932 wait complete. 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.293+0000 7f5f35b90640 1 Processor -- start 2024-03-29T04:10:50.273 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.294+0000 7f5f35b90640 1 -- start start 2024-03-29T04:10:50.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f180051c0 0x7f5f301571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.275 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 0x7f5f3014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3015f500 con 0x7f5f3014b5b0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39872/0 (socket says 172.21.15.181:39872) 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 -- 172.21.15.181:0/829822112 learned_addr learned my addr 172.21.15.181:0/829822112 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30162eb0 con 0x7f5f3006cac0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3015f680 con 0x7f5f180051c0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 0x7f5f3014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 -- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f180051c0 msgr2=0x7f5f301571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f180051c0 0x7f5f301571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 -- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 msgr2=0x7f5f3014bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 0x7f5f3014bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.295+0000 7f5f2effd640 1 -- 172.21.15.181:0/829822112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3014bd40 con 0x7f5f3014b5b0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.296+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 0x7f5f3014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.296+0000 7f5f2effd640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400d4f0 tx=0x7f5f2400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.276 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.296+0000 7f5f17fff640 1 -- 172.21.15.181:0/829822112 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24005230 con 0x7f5f3014b5b0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.296+0000 7f5f17fff640 1 -- 172.21.15.181:0/829822112 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240053d0 con 0x7f5f3014b5b0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.297+0000 7f5f17fff640 1 -- 172.21.15.181:0/829822112 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24012840 con 0x7f5f3014b5b0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.297+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f30158920 con 0x7f5f3014b5b0 2024-03-29T04:10:50.277 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.297+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f30157b60 con 0x7f5f3014b5b0 2024-03-29T04:10:50.278 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.299+0000 7f5f17fff640 1 -- 172.21.15.181:0/829822112 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f240129e0 con 0x7f5f3014b5b0 2024-03-29T04:10:50.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.300+0000 7f5f17fff640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 0x7f5f1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.300+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 0x7f5f1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.300+0000 7f5f17fff640 1 -- 172.21.15.181:0/829822112 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f2408b780 con 0x7f5f3014b5b0 2024-03-29T04:10:50.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.301+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 0x7f5f1c06e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001eb0 tx=0x7f5f20001170 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.301+0000 7f5f38780740 1 Processor -- start 2024-03-29T04:10:50.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.301+0000 7f5f38780740 1 -- start start 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426da86e0 0x560426dd84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560426d12880 con 0x560426dd5200 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560426cba8f0 con 0x560426dd55e0 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560426d0e830 con 0x560426da86e0 2024-03-29T04:10:50.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39888/0 (socket says 172.21.15.181:39888) 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/1935390531 learned_addr learned my addr 172.21.15.181:0/1935390531 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2effd640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426da86e0 0x560426dd84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426da86e0 msgr2=0x560426dd84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426da86e0 0x560426dd84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 msgr2=0x560426dd7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.302+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/1935390531 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560426cc5140 con 0x560426dd5200 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.303+0000 7f5f2effd640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.303+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f20012180 tx=0x7f5f20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e7929100f45d3c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.303+0000 7f5f157fa640 1 -- 172.21.15.181:0/1935390531 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001c430 con 0x560426dd5200 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.303+0000 7f5f157fa640 1 -- 172.21.15.181:0/1935390531 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2001c5d0 con 0x560426dd5200 2024-03-29T04:10:50.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.303+0000 7f5f157fa640 1 -- 172.21.15.181:0/1935390531 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001c980 con 0x560426dd5200 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 -- 172.21.15.181:0/1935390531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 msgr2=0x560426da81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5f20012180 tx=0x7f5f20019000 comp rx=0 tx=0).stop 2024-03-29T04:10:50.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 -- 172.21.15.181:0/1935390531 shutdown_connections 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426da86e0 0x560426dd84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 --2- 172.21.15.181:0/1935390531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd5200 0x560426da81a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 -- 172.21.15.181:0/1935390531 >> 172.21.15.181:0/1935390531 conn(0x560426daa410 msgr2=0x560426df2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 -- 172.21.15.181:0/1935390531 shutdown_connections 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.304+0000 7f5f38780740 1 -- 172.21.15.181:0/1935390531 wait complete. 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.305+0000 7f5f38780740 1 -- start start 2024-03-29T04:10:50.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.305+0000 7f5f38780740 1 Processor -- start 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426dc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560426dacab0 con 0x560426dd9790 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560426ccdcf0 con 0x560426dd55e0 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560426daf2f0 con 0x560426dd5200 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f38780740 10 client.0 _ll_register_callbacks cb 0x560426cbff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:45106/0 (socket says 172.21.15.181:45106) 2024-03-29T04:10:50.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/2659144827 learned_addr learned my addr 172.21.15.181:0/2659144827 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f2effd640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.306+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426dc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 msgr2=0x560426dd9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 msgr2=0x560426dc8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426dc8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/2659144827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x560426af07d0 con 0x560426dd5200 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2effd640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001dd40 tx=0x7f5f20001a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daad665cf1849eae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3301/0 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001c430 con 0x560426dd5200 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mon.2 v2:172.21.15.187:3301/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f20026050 con 0x560426dd5200 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2001c5d0 con 0x560426dd5200 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5f2001e630 con 0x560426dd5200 2024-03-29T04:10:50.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f38780740 10 client.5129 fetch_fsmap learned FSMap version 18 2024-03-29T04:10:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.307+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x560426a263f0 con 0x560426dd5200 2024-03-29T04:10:50.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mon.2 v2:172.21.15.187:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2001c5d0 con 0x560426dd5200 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mon.2 v2:172.21.15.187:3301/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5f2003c050 con 0x560426dd5200 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 10 client.5129 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 10 client.5129 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 20 client.5129 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426df1010 0x560426df1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 4 client.5129 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.308+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560426d1fdc0 con 0x560426df1010 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.309+0000 7f5f2effd640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426df1010 0x560426df1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.309+0000 7f5f2effd640 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426df1010 0x560426df1440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332433bdcef1e6ae server_cookie=52b4c4a368e1dd32 in_seq=0 out_seq=0 2024-03-29T04:10:50.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.309+0000 7f5f0e7fc640 10 client.5129 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.311+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/2659144827 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560426d1fdc0 con 0x560426df1010 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.311+0000 7f5f0e7fc640 10 client.5129 handle_command_reply: tid=0 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.311+0000 7f5f38780740 1 client.5129 shutdown 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 msgr2=0x560426ddbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001dd40 tx=0x7f5f20001a20 comp rx=0 tx=0).stop 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 shutdown_connections 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426df1010 0x560426df1440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426dc8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd55e0 0x560426dd9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 --2- 172.21.15.181:0/2659144827 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd5200 0x560426ddbac0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 >> 172.21.15.181:0/2659144827 conn(0x560426da86e0 msgr2=0x560426dac0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f0e7fc640 0 client.5129 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 shutdown_connections 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 1 -- 172.21.15.181:0/2659144827 wait complete. 2024-03-29T04:10:50.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.312+0000 7f5f38780740 20 client.5129 trim_cache size 0 max 16384 2024-03-29T04:10:50.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.339+0000 7f5f38780740 1 Processor -- start 2024-03-29T04:10:50.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.339+0000 7f5f38780740 1 -- start start 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426df4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301fd270 con 0x560426dd9790 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301fc250 con 0x560426dd5200 2024-03-29T04:10:50.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3014e5d0 con 0x560426dd55e0 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426df4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:39022/0 (socket says 172.21.15.181:39022) 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2effd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:45112/0 (socket says 172.21.15.181:45112) 2024-03-29T04:10:50.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/981254753 learned_addr learned my addr 172.21.15.181:0/981254753 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 msgr2=0x560426df5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/981254753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 msgr2=0x560426df4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426df4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 -- 172.21.15.181:0/981254753 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560426ddd9e0 con 0x560426dd5200 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2effd640 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.341+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001b50 tx=0x7f5f20022050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ceebe2da854f30a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.342+0000 7f5f14ff9640 1 -- 172.21.15.181:0/981254753 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001d240 con 0x560426dd5200 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.342+0000 7f5f14ff9640 1 -- 172.21.15.181:0/981254753 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2001d3e0 con 0x560426dd5200 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.342+0000 7f5f14ff9640 1 -- 172.21.15.181:0/981254753 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001d5b0 con 0x560426dd5200 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 -- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 msgr2=0x560426df0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001b50 tx=0x7f5f20022050 comp rx=0 tx=0).stop 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 -- 172.21.15.181:0/981254753 shutdown_connections 2024-03-29T04:10:50.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426df4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426df5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 --2- 172.21.15.181:0/981254753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426df0240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 -- 172.21.15.181:0/981254753 >> 172.21.15.181:0/981254753 conn(0x560426da86e0 msgr2=0x560426dc7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 -- 172.21.15.181:0/981254753 shutdown_connections 2024-03-29T04:10:50.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.343+0000 7f5f38780740 1 -- 172.21.15.181:0/981254753 wait complete. 2024-03-29T04:10:50.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.344+0000 7f5f38780740 1 -- start start 2024-03-29T04:10:50.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.344+0000 7f5f38780740 1 Processor -- start 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426e5f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426dd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301fbc20 con 0x560426dd9790 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560426e61b20 con 0x560426dd5200 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560426da7150 con 0x560426dd55e0 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f38780740 10 client.0 _ll_register_callbacks cb 0x7f5f24055840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:39906/0 (socket says 172.21.15.181:39906) 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 -- 172.21.15.181:0/515223267 learned_addr learned my addr 172.21.15.181:0/515223267 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2effd640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426dd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426e5f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 -- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 msgr2=0x560426dd7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426dd7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.329 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 -- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 msgr2=0x560426e5f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426e5f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2ffff640 1 -- 172.21.15.181:0/515223267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604269e60f0 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426e5f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.345+0000 7f5f2effd640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426dd7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f2ffff640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e200 tx=0x7f5f28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de987dfb608f9204 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2804f060 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5f28035e70 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2001d3e0 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f5f28055050 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 10 client.5646 fetch_fsmap learned FSMap version 18 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x560426e61860 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2001d3e0 con 0x560426dd9790 2024-03-29T04:10:50.330 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f5f28058020 con 0x560426dd9790 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 10 client.5646 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 10 client.5646 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 20 client.5646 populate_metadata read hostname 'smithi181' 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426e5d0e0 0x560426e0cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 4 client.5646 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560426d1a110 con 0x560426e5d0e0 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.346+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426e5d0e0 0x560426e0cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f0e7fc640 10 client.5646 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f2f7fe640 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426e5d0e0 0x560426e0cb50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3577c0e3bc54b20 server_cookie=7026a3e3310a43e6 in_seq=0 out_seq=0 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f0e7fc640 1 -- 172.21.15.181:0/515223267 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x560426d1a110 con 0x560426e5d0e0 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f0e7fc640 10 client.5646 handle_command_reply: tid=0 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 client.5646 shutdown 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 msgr2=0x560426e5cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800e200 tx=0x7f5f28004ab0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 shutdown_connections 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x560426e5d0e0 0x560426e0cb50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560426dd9790 0x560426e5cba0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560426dd55e0 0x560426dd7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 --2- 172.21.15.181:0/515223267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560426dd5200 0x560426e5f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 >> 172.21.15.181:0/515223267 conn(0x560426da86e0 msgr2=0x560426defee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f0e7fc640 0 client.5646 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 shutdown_connections 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 1 -- 172.21.15.181:0/515223267 wait complete. 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.347+0000 7f5f38780740 20 client.5646 trim_cache size 0 max 16384 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": { 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stdout: "2842fbf9-7715-41c9-99ad-42a9c182c2f2": { 2024-03-29T04:10:50.332 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/", 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "2842fbf9-7715-41c9-99ad-42a9c182c2f2", 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "options": "recursive" 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_stats": { 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "2842fbf9-7715-41c9-99ad-42a9c182c2f2": { 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "start_time": "2024-03-29T04:10:50.119864+0000", 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/", 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_started": 0, 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_passed": 0, 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_failed": 0, 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_skipped": 0 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:10:50.333 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:10:50.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 msgr2=0x7f5f1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 0x7f5f1c06e9f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001eb0 tx=0x7f5f20001170 comp rx=0 tx=0).stop 2024-03-29T04:10:50.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 msgr2=0x7f5f3006c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:10:50.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400d4f0 tx=0x7f5f2400d9b0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.334 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 shutdown_connections 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3006cac0 0x7f5f3014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f3014b5b0 0x7f5f3006c580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f5f1c06c530 0x7f5f1c06e9f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 --2- 172.21.15.181:0/829822112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f180051c0 0x7f5f301571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 >> 172.21.15.181:0/829822112 conn(0x7f5f300d0840 msgr2=0x7f5f300b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 shutdown_connections 2024-03-29T04:10:50.335 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:10:50.348+0000 7f5f35b90640 1 -- 172.21.15.181:0/829822112 wait complete. 2024-03-29T04:10:50.338 INFO:tasks.cephfs.filesystem:scrub status for tag:2842fbf9-7715-41c9-99ad-42a9c182c2f2 - {'path': '/', 'tag': '2842fbf9-7715-41c9-99ad-42a9c182c2f2', 'options': 'recursive'} 2024-03-29T04:11:12.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:12.061 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:20.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:20.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-29T04:11:20.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 Processor -- start 2024-03-29T04:11:20.451 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 -- start start 2024-03-29T04:11:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19280afd60 0x7f192815f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 0x7f19280b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 0x7f192806e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f192806fac0 con 0x7f19280afd60 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1928162610 con 0x7f1928162070 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f192dace640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19281628f0 con 0x7f19280b3c90 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.472+0000 7f19277fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19280afd60 0x7f192815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19280afd60 0x7f192815f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55852/0 (socket says 172.21.15.181:55852) 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 -- 172.21.15.181:0/3062047527 learned_addr learned my addr 172.21.15.181:0/3062047527 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f1927fff640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 0x7f192806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f191effd640 1 --2- 172.21.15.181:0/3062047527 >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 0x7f19280b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 -- 172.21.15.181:0/3062047527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 msgr2=0x7f192806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 0x7f192806e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 -- 172.21.15.181:0/3062047527 >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 msgr2=0x7f19280b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 --2- 172.21.15.181:0/3062047527 >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 0x7f19280b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 -- 172.21.15.181:0/3062047527 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1928162bd0 con 0x7f19280afd60 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f191effd640 1 --2- 172.21.15.181:0/3062047527 >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 0x7f19280b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f1927fff640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 0x7f192806e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:20.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.473+0000 7f19277fe640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280afd60 0x7f192815f500 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f192806d060 tx=0x7f191802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a69bc601df7ed45 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f1926ffd640 1 -- 172.21.15.181:0/3062047527 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918035020 con 0x7f19280afd60 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f1926ffd640 1 -- 172.21.15.181:0/3062047527 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918031870 con 0x7f19280afd60 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f1926ffd640 1 -- 172.21.15.181:0/3062047527 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918031c40 con 0x7f19280afd60 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 -- 172.21.15.181:0/3062047527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280afd60 msgr2=0x7f192815f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280afd60 0x7f192815f500 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f192806d060 tx=0x7f191802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 -- 172.21.15.181:0/3062047527 shutdown_connections 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280b3c90 0x7f192806e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 --2- 172.21.15.181:0/3062047527 >> v2:172.21.15.187:3300/0 conn(0x7f1928162070 0x7f19280b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 --2- 172.21.15.181:0/3062047527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280afd60 0x7f192815f500 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 -- 172.21.15.181:0/3062047527 >> 172.21.15.181:0/3062047527 conn(0x7f19280d0840 msgr2=0x7f19280d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.474+0000 7f192dace640 1 -- 172.21.15.181:0/3062047527 shutdown_connections 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.475+0000 7f192dace640 1 -- 172.21.15.181:0/3062047527 wait complete. 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 Processor -- start 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 -- start start 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280afd60 0x7f1928155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1928162070 0x7f19281e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f192806e5d0 con 0x7f19280b3c90 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1928162eb0 con 0x7f1928162070 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.476+0000 7f192dace640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f192806e750 con 0x7f19280afd60 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55868/0 (socket says 172.21.15.181:55868) 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 -- 172.21.15.181:0/3659589296 learned_addr learned my addr 172.21.15.181:0/3659589296 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f19277fe640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280afd60 0x7f1928155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f1927fff640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1928162070 0x7f19281e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 -- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280afd60 msgr2=0x7f1928155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280afd60 0x7f1928155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 -- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1928162070 msgr2=0x7f19281e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1928162070 0x7f19281e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 -- 172.21.15.181:0/3659589296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1928156f00 con 0x7f19280b3c90 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f191effd640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f191000c820 tx=0x7f191000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f1924ff9640 1 -- 172.21.15.181:0/3659589296 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910015070 con 0x7f19280b3c90 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f1924ff9640 1 -- 172.21.15.181:0/3659589296 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19100079d0 con 0x7f19280b3c90 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1928155e60 con 0x7f19280b3c90 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f1924ff9640 1 -- 172.21.15.181:0/3659589296 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910007d80 con 0x7f19280b3c90 2024-03-29T04:11:20.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.477+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1928156700 con 0x7f19280b3c90 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.479+0000 7f1924ff9640 1 -- 172.21.15.181:0/3659589296 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1910018070 con 0x7f19280b3c90 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.479+0000 7f1924ff9640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 0x7f190806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.479+0000 7f1924ff9640 1 -- 172.21.15.181:0/3659589296 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f191008b6b0 con 0x7f19280b3c90 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.479+0000 7f19277fe640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 0x7f190806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.480+0000 7f19277fe640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 0x7f190806e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1918036000 tx=0x7f191803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.481+0000 7f193078b740 1 Processor -- start 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.481+0000 7f193078b740 1 -- start start 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79ddd6e0 0x556e79e0d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e79d47880 con 0x556e79e0a200 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e79cef8f0 con 0x556e79e0a5e0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556e79d43830 con 0x556e79ddd6e0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55884/0 (socket says 172.21.15.181:55884) 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 -- 172.21.15.181:0/1938645692 learned_addr learned my addr 172.21.15.181:0/1938645692 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f1927fff640 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79ddd6e0 0x556e79e0d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f191effd640 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 -- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79ddd6e0 msgr2=0x556e79e0d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79ddd6e0 0x556e79e0d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 -- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 msgr2=0x556e79e0cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e0cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.482+0000 7f19277fe640 1 -- 172.21.15.181:0/1938645692 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e79cfa140 con 0x556e79e0a200 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f19277fe640 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1918009930 tx=0x7f1918030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb14f0f1cd6e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f191d7fa640 1 -- 172.21.15.181:0/1938645692 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918052070 con 0x556e79e0a200 2024-03-29T04:11:20.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f191d7fa640 1 -- 172.21.15.181:0/1938645692 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918044e50 con 0x556e79e0a200 2024-03-29T04:11:20.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f191d7fa640 1 -- 172.21.15.181:0/1938645692 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918035040 con 0x556e79e0a200 2024-03-29T04:11:20.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f193078b740 1 -- 172.21.15.181:0/1938645692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 msgr2=0x556e79ddd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f193078b740 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1918009930 tx=0x7f1918030000 comp rx=0 tx=0).stop 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.483+0000 7f193078b740 1 -- 172.21.15.181:0/1938645692 shutdown_connections 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79ddd6e0 0x556e79e0d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 --2- 172.21.15.181:0/1938645692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79ddd1a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 -- 172.21.15.181:0/1938645692 >> 172.21.15.181:0/1938645692 conn(0x556e79ddf410 msgr2=0x556e79e27f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 -- 172.21.15.181:0/1938645692 shutdown_connections 2024-03-29T04:11:20.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 -- 172.21.15.181:0/1938645692 wait complete. 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 -- start start 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.484+0000 7f193078b740 1 Processor -- start 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e28460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e3b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e79de1ab0 con 0x556e79e289a0 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e79d02cf0 con 0x556e79e0a5e0 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556e79de42f0 con 0x556e79e0a200 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f193078b740 10 client.0 _ll_register_callbacks cb 0x556e79cf4f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55892/0 (socket says 172.21.15.181:55892) 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 -- 172.21.15.181:0/412405382 learned_addr learned my addr 172.21.15.181:0/412405382 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f19277fe640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e28460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f191effd640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e3b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 -- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 msgr2=0x556e79e28460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e28460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 -- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 msgr2=0x556e79e3b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e3b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 -- 172.21.15.181:0/412405382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e79b257d0 con 0x556e79e289a0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f19277fe640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e28460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f191effd640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e3b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f1927fff640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f192000df10 tx=0x7f192000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7703d0795aca31f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.485+0000 7f19067fc640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192001c060 con 0x556e79e289a0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1920003b90 con 0x556e79e289a0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1918044e50 con 0x556e79e289a0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1920022050 con 0x556e79e289a0 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 10 client.5658 fetch_fsmap learned FSMap version 18 2024-03-29T04:11:20.476 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556e79a5b3f0 con 0x556e79e289a0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918044e50 con 0x556e79e289a0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f1920025020 con 0x556e79e289a0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 10 client.5658 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 10 client.5658 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 20 client.5658 populate_metadata read hostname 'smithi181' 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79e0be30 0x556e79e0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 4 client.5658 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e79d54dc0 con 0x556e79e0be30 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.486+0000 7f19277fe640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79e0be30 0x556e79e0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f19067fc640 10 client.5658 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f19277fe640 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79e0be30 0x556e79e0c240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec59bbd23d108546 server_cookie=2375dfb0118a608c in_seq=0 out_seq=0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f19067fc640 1 -- 172.21.15.181:0/412405382 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556e79d54dc0 con 0x556e79e0be30 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f19067fc640 10 client.5658 handle_command_reply: tid=0 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 client.5658 shutdown 2024-03-29T04:11:20.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 msgr2=0x556e79e0e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f192000df10 tx=0x7f192000c7f0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 shutdown_connections 2024-03-29T04:11:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79e0be30 0x556e79e0c240 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e289a0 0x556e79e0e130 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79e3b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 --2- 172.21.15.181:0/412405382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e28460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 >> 172.21.15.181:0/412405382 conn(0x556e79ddd6e0 msgr2=0x556e79de10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f19067fc640 0 client.5658 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 shutdown_connections 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 1 -- 172.21.15.181:0/412405382 wait complete. 2024-03-29T04:11:20.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.487+0000 7f193078b740 20 client.5658 trim_cache size 0 max 16384 2024-03-29T04:11:20.484 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.505+0000 7f193078b740 1 Processor -- start 2024-03-29T04:11:20.484 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.505+0000 7f193078b740 1 -- start start 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e2b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e289a0 0x556e79e089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1928201130 con 0x556e79e0a5e0 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1928200150 con 0x556e79e289a0 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19281513e0 con 0x556e79e0a200 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f191effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f191effd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55898/0 (socket says 172.21.15.181:55898) 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f191effd640 1 -- 172.21.15.181:0/4262803443 learned_addr learned my addr 172.21.15.181:0/4262803443 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f19277fe640 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e2b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.506+0000 7f1927fff640 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e289a0 0x556e79e089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 -- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 msgr2=0x556e79e2b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e2b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 -- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e289a0 msgr2=0x556e79e089e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e289a0 0x556e79e089e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 -- 172.21.15.181:0/4262803443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e79a1b0f0 con 0x556e79e0a5e0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191effd640 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f19100558a0 tx=0x7f19100557e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bace8115ad11cc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191cff9640 1 -- 172.21.15.181:0/4262803443 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191001f040 con 0x556e79e0a5e0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191cff9640 1 -- 172.21.15.181:0/4262803443 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1910089e50 con 0x556e79e0a5e0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.507+0000 7f191cff9640 1 -- 172.21.15.181:0/4262803443 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910055ab0 con 0x556e79e0a5e0 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- 172.21.15.181:0/4262803443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 msgr2=0x556e79e084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f19100558a0 tx=0x7f19100557e0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- 172.21.15.181:0/4262803443 shutdown_connections 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e289a0 0x556e79e089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a5e0 0x556e79e084a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 --2- 172.21.15.181:0/4262803443 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e0a200 0x556e79e2b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- 172.21.15.181:0/4262803443 >> 172.21.15.181:0/4262803443 conn(0x556e79ddd6e0 msgr2=0x556e79e021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- 172.21.15.181:0/4262803443 shutdown_connections 2024-03-29T04:11:20.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- 172.21.15.181:0/4262803443 wait complete. 2024-03-29T04:11:20.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 -- start start 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.508+0000 7f193078b740 1 Processor -- start 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79ddc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e289a0 0x556e79ddcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e79dddb20 con 0x556e79e0a200 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1928151a30 con 0x556e79e0a5e0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556e79ea5100 con 0x556e79e289a0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f193078b740 10 client.0 _ll_register_callbacks cb 0x7f192000db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55906/0 (socket says 172.21.15.181:55906) 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 -- 172.21.15.181:0/3630221077 learned_addr learned my addr 172.21.15.181:0/3630221077 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f1927fff640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e289a0 0x556e79ddcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f191effd640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79ddc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 -- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e289a0 msgr2=0x556e79ddcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e289a0 0x556e79ddcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 -- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 msgr2=0x556e79ddc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79ddc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f19277fe640 1 -- 172.21.15.181:0/3630221077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e79ea3ec0 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.509+0000 7f191effd640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79ddc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19277fe640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f19180090b0 tx=0x7f191802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdd8c9cf701cb4d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918052040 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mon.0 v2:172.21.15.181:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f19180319b0 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1910089e50 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f191806a050 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 10 client.5664 fetch_fsmap learned FSMap version 18 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x556e79df0d30 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mon.0 v2:172.21.15.181:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1910089e50 con 0x556e79e0a200 2024-03-29T04:11:20.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mon.0 v2:172.21.15.181:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f191806d020 con 0x556e79e0a200 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 10 client.5664 fetch_fsmap finished waiting for FSMap version 18 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 10 client.5664 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 20 client.5664 populate_metadata read hostname 'smithi181' 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79df1390 0x556e79e08780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 4 client.5664 mds_command: new command op to 5373 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 --> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e79d4f110 con 0x556e79df1390 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.510+0000 7f191effd640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79df1390 0x556e79e08780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f19067fc640 10 client.5664 ms_handle_connect on v2:172.21.15.181:6835/2447436157 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f191effd640 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79df1390 0x556e79e08780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab6580ec7c5e0dc server_cookie=e387d32f4657d41c in_seq=0 out_seq=0 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f19067fc640 1 -- 172.21.15.181:0/3630221077 <== mds.0 v2:172.21.15.181:6835/2447436157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x556e79d4f110 con 0x556e79df1390 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f19067fc640 10 client.5664 handle_command_reply: tid=0 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 client.5664 shutdown 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 msgr2=0x556e79e9dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f19180090b0 tx=0x7f191802e000 comp rx=0 tx=0).stop 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 shutdown_connections 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:6835/2447436157,v1:172.21.15.181:6837/2447436157] conn(0x556e79df1390 0x556e79e08780 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556e79e289a0 0x556e79ddcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f19067fc640 0 client.5664 ms_handle_reset on v2:172.21.15.181:6835/2447436157 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e79e0a5e0 0x556e79ddc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 --2- 172.21.15.181:0/3630221077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e79e0a200 0x556e79e9dc60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 >> 172.21.15.181:0/3630221077 conn(0x556e79ddd6e0 msgr2=0x556e79dd8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 shutdown_connections 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 1 -- 172.21.15.181:0/3630221077 wait complete. 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f193078b740 20 client.5664 trim_cache size 0 max 16384 2024-03-29T04:11:20.496 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "no active scrubs running", 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "scrubs": {}, 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_stats": { 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "2842fbf9-7715-41c9-99ad-42a9c182c2f2": { 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "start_time": "2024-03-29T04:10:50.119864+0000", 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "path": "/", 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_started": 75, 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_passed": 75, 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_failed": 0, 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: "uninline_skipped": 0 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-03-29T04:11:20.497 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:20.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 msgr2=0x7f190806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.511+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 0x7f190806e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1918036000 tx=0x7f191803c040 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 msgr2=0x7f1928154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f191000c820 tx=0x7f191000cce0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 shutdown_connections 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1928162070 0x7f19281e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19280b3c90 0x7f1928154150 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f190806c530 0x7f190806e9f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 --2- 172.21.15.181:0/3659589296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19280afd60 0x7f1928155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 >> 172.21.15.181:0/3659589296 conn(0x7f19280d0840 msgr2=0x7f19280ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 shutdown_connections 2024-03-29T04:11:20.499 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:20.512+0000 7f192dace640 1 -- 172.21.15.181:0/3659589296 wait complete. 2024-03-29T04:11:20.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T04:11:20.502 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-29T04:11:20.502 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-29T04:11:20.502 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-29T04:11:20.517 INFO:teuthology.orchestra.run.smithi181.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-29T04:11:20.518 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-29T04:11:20.518 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-29T04:11:20.573 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-29T04:11:20.573 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:11:20.573 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:11:20.627 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-29T04:11:20.628 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-29T04:11:20.628 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-29T04:11:20.683 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-29T04:11:20.683 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:11:20.683 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:11:20.738 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-29T04:11:20.739 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-29T04:11:20.739 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-29T04:11:20.795 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-29T04:11:20.795 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:11:20.795 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:11:20.850 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-29T04:11:20.851 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:11:20.851 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:11:20.906 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-29T04:11:20.906 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-29T04:11:20.906 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-29T04:11:20.961 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-29T04:11:20.962 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-29T04:11:20.962 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-29T04:11:21.016 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-29T04:11:21.017 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-29T04:11:21.017 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-29T04:11:21.072 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-29T04:11:21.073 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:11:21.073 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:11:21.127 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-29T04:11:21.127 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-29T04:11:21.127 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-29T04:11:21.182 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-29T04:11:21.182 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:11:21.182 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:11:21.237 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-29T04:11:21.237 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:11:21.238 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:11:21.293 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-29T04:11:21.293 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:11:21.293 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:11:21.348 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-29T04:11:21.348 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:11:21.348 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:11:21.403 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-29T04:11:21.403 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:11:21.403 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:11:21.458 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-29T04:11:21.458 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:11:21.458 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:11:21.513 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-29T04:11:21.513 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:11:21.513 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:11:21.567 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-29T04:11:21.568 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-29T04:11:21.568 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-29T04:11:21.622 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-29T04:11:21.623 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:11:21.623 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:11:21.677 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-29T04:11:21.677 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:11:21.677 DEBUG:teuthology.orchestra.run.smithi181:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:11:21.731 INFO:teuthology.orchestra.run.smithi181.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-29T04:11:21.731 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-29T04:11:21.732 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-29T04:11:21.732 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-29T04:11:21.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T04:11:21.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:11:21.817 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T04:11:21.870 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T04:11:21.870 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:11:21.870 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:11:21.870 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:21.885 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:11:21.886 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:21.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:22.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.123+0000 7f9b4f315640 1 Processor -- start 2024-03-29T04:11:22.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.123+0000 7f9b4f315640 1 -- start start 2024-03-29T04:11:22.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4f315640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 0x7f9b48071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4f315640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b480cb580 0x7f9b480cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4f315640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b480724e0 con 0x7f9b480cd3b0 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4f315640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f9b480730f0 con 0x7f9b480cb580 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4f315640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b480727c0 con 0x7f9b4815a960 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d8b2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b480cb580 0x7f9b480cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4c8b0640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9b480cd3b0 0x7f9b480cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36074/0 (socket says 172.21.15.181:36074) 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4c8b0640 1 -- 172.21.15.181:0/710152210 learned_addr learned my addr 172.21.15.181:0/710152210 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d0b1640 1 --2- 172.21.15.181:0/710152210 >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 0x7f9b48071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d8b2640 1 -- 172.21.15.181:0/710152210 >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 msgr2=0x7f9b48071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d8b2640 1 --2- 172.21.15.181:0/710152210 >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 0x7f9b48071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d8b2640 1 -- 172.21.15.181:0/710152210 >> v1:172.21.15.181:6789/0 conn(0x7f9b480cd3b0 legacy=0x7f9b480cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d8b2640 1 -- 172.21.15.181:0/710152210 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48072aa0 con 0x7f9b480cb580 2024-03-29T04:11:22.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.124+0000 7f9b4d0b1640 1 --2- 172.21.15.181:0/710152210 >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 0x7f9b48071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.125+0000 7f9b4d8b2640 1 --2- 172.21.15.181:0/710152210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cb580 0x7f9b480cddd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9b480691c0 tx=0x7f9b4002cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23538c6bf70282c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.125+0000 7f9b3ffff640 1 -- 172.21.15.181:0/710152210 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4002db70 con 0x7f9b480cb580 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.125+0000 7f9b3ffff640 1 -- 172.21.15.181:0/710152210 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4002dd10 con 0x7f9b480cb580 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.125+0000 7f9b3ffff640 1 -- 172.21.15.181:0/710152210 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40008660 con 0x7f9b480cb580 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 -- 172.21.15.181:0/710152210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cb580 msgr2=0x7f9b480cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 --2- 172.21.15.181:0/710152210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cb580 0x7f9b480cddd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9b480691c0 tx=0x7f9b4002cfd0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 -- 172.21.15.181:0/710152210 shutdown_connections 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 --2- 172.21.15.181:0/710152210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cb580 0x7f9b480cddd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 --2- 172.21.15.181:0/710152210 >> v2:172.21.15.187:3300/0 conn(0x7f9b4815a960 0x7f9b48071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 -- 172.21.15.181:0/710152210 >> 172.21.15.181:0/710152210 conn(0x7f9b480b1c90 msgr2=0x7f9b480b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 -- 172.21.15.181:0/710152210 shutdown_connections 2024-03-29T04:11:22.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.126+0000 7f9b4f315640 1 -- 172.21.15.181:0/710152210 wait complete. 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.127+0000 7f9b4f315640 1 Processor -- start 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.127+0000 7f9b4f315640 1 -- start start 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b480cb580 0x7f9b4814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4815a960 0x7f9b4815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48071fb0 con 0x7f9b480cd3b0 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4815ad40 con 0x7f9b4815a960 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4f315640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b48072130 con 0x7f9b480cb580 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55924/0 (socket says 172.21.15.181:55924) 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 -- 172.21.15.181:0/4151846646 learned_addr learned my addr 172.21.15.181:0/4151846646 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4d0b1640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b480cb580 0x7f9b4814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4d8b2640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4815a960 0x7f9b4815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 -- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b480cb580 msgr2=0x7f9b4814fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b480cb580 0x7f9b4814fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 -- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4815a960 msgr2=0x7f9b4815b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4815a960 0x7f9b4815b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 -- 172.21.15.181:0/4151846646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48154790 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.128+0000 7f9b4c8b0640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c400 tx=0x7f9b3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.129+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38015020 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.129+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b38005ce0 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.129+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b48154a70 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.129+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380072f0 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.129+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b48155030 con 0x7f9b480cd3b0 2024-03-29T04:11:22.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.130+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48068d40 con 0x7f9b480cd3b0 2024-03-29T04:11:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.130+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b38018020 con 0x7f9b480cd3b0 2024-03-29T04:11:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.130+0000 7f9b3dffb640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 0x7f9b2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.131+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9b3808d230 con 0x7f9b480cd3b0 2024-03-29T04:11:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.131+0000 7f9b4d0b1640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 0x7f9b2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.131+0000 7f9b4d0b1640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 0x7f9b2c06e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b4815bb40 tx=0x7f9b44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.135+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b48068d40 con 0x7f9b480cd3b0 2024-03-29T04:11:22.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.317+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b48075a10 con 0x7f9b480cd3b0 2024-03-29T04:11:22.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.318+0000 7f9b3dffb640 1 -- 172.21.15.181:0/4151846646 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7f9b3801f030 con 0x7f9b480cd3b0 2024-03-29T04:11:22.298 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:22.298 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":41,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:58.390325+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 msgr2=0x7f9b2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 0x7f9b2c06e9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9b4815bb40 tx=0x7f9b44008040 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 msgr2=0x7f9b4815afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c400 tx=0x7f9b3800c8c0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 shutdown_connections 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4815a960 0x7f9b4815b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480cd3b0 0x7f9b4815afa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b2c06c530 0x7f9b2c06e9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 --2- 172.21.15.181:0/4151846646 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b480cb580 0x7f9b4814fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 >> 172.21.15.181:0/4151846646 conn(0x7f9b480b1c90 msgr2=0x7f9b480749e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 shutdown_connections 2024-03-29T04:11:22.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.320+0000 7f9b4f315640 1 -- 172.21.15.181:0/4151846646 wait complete. 2024-03-29T04:11:22.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:10:20.863137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:10:21.635882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:11:22.311 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:22.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 Processor -- start 2024-03-29T04:11:22.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 -- start start 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f53340b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53340b66e0 con 0x7f53340ba090 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53340d2ff0 con 0x7f53340b73d0 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.437+0000 7f533a396640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53340d2d90 con 0x7f53340b5dc0 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f53337fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f53340ba090 0x7f53340d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36080/0 (socket says 172.21.15.181:36080) 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f53337fe640 1 -- 172.21.15.181:0/2080223811 learned_addr learned my addr 172.21.15.181:0/2080223811 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5333fff640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f53340b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532102692 0 0) 0x7f53340b66e0 con 0x7f53340ba090 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f531c003520 con 0x7f53340ba090 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1858280821 0 0) 0x7f531c003520 con 0x7f53340ba090 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 msgr2=0x7f53340b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f53340b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 >> v1:172.21.15.187:6789/0 conn(0x7f53340b73d0 legacy=0x7f53340b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53340d32d0 con 0x7f53340ba090 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f5328001670 con 0x7f53340ba090 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 >> v1:172.21.15.181:6789/0 conn(0x7f53340ba090 legacy=0x7f53340d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f531c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c003f20 0x7f531c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c004d40 0x7f531c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53340d2d90 con 0x7f531c004d40 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f531c00ba20 con 0x7f531c003f20 2024-03-29T04:11:22.420 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f531c00bc20 con 0x7f53340b5dc0 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5333fff640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f531c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f5338933640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c004d40 0x7f531c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.438+0000 7f53337fe640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c003f20 0x7f531c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5338933640 1 -- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 msgr2=0x7f531c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5338933640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f531c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5338933640 1 -- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c003f20 msgr2=0x7f531c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5338933640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f531c003f20 0x7f531c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5338933640 1 -- 172.21.15.181:0/2080223811 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c003520 con 0x7f531c004d40 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.439+0000 7f5333fff640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53340b5dc0 0x7f531c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.440+0000 7f5338933640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c004d40 0x7f531c0051c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f532c002010 tx=0x7f531c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a054dc19ddebc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.440+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c039070 con 0x7f531c004d40 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.440+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532c02c750 con 0x7f531c004d40 2024-03-29T04:11:22.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.440+0000 7f5332ffd640 1 -- 172.21.15.181:0/2080223811 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c001be0 con 0x7f531c004d40 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.440+0000 7f533a396640 1 -- 172.21.15.181:0/2080223811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c004d40 msgr2=0x7f531c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f533a396640 1 --2- 172.21.15.181:0/2080223811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c004d40 0x7f531c0051c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f532c002010 tx=0x7f531c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f5338933640 1 -- 172.21.15.181:0/2080223811 reap_dead start 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f533a396640 1 -- 172.21.15.181:0/2080223811 shutdown_connections 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f533a396640 1 -- 172.21.15.181:0/2080223811 >> 172.21.15.181:0/2080223811 conn(0x7f53340678f0 msgr2=0x7f53340d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f533a396640 1 -- 172.21.15.181:0/2080223811 shutdown_connections 2024-03-29T04:11:22.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.441+0000 7f533a396640 1 -- 172.21.15.181:0/2080223811 wait complete. 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.442+0000 7f533a396640 1 Processor -- start 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.442+0000 7f533a396640 1 -- start start 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.443+0000 7f533a396640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c004d40 0x7f533415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.443+0000 7f533a396640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.443+0000 7f533a396640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 0x7f53340addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.443+0000 7f533a396640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53340d3b70 con 0x7f533415f8d0 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.443+0000 7f533a396640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53340d3890 con 0x7f53340ad940 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f533a396640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53340d4090 con 0x7f531c004d40 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55946/0 (socket says 172.21.15.181:55946) 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 -- 172.21.15.181:0/3841592718 learned_addr learned my addr 172.21.15.181:0/3841592718 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5338933640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 0x7f53340addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5333fff640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c004d40 0x7f533415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 -- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c004d40 msgr2=0x7f533415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c004d40 0x7f533415c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 -- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 msgr2=0x7f53340addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 0x7f53340addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 -- 172.21.15.181:0/3841592718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533415cea0 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5338933640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 0x7f53340addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f53337fe640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f53280072c0 tx=0x7f5328007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800f960 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5328018070 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800fd80 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5334160440 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.444+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5334160a00 con 0x7f533415f8d0 2024-03-29T04:11:22.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.445+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53340749e0 con 0x7f533415f8d0 2024-03-29T04:11:22.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.446+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f532801d460 con 0x7f533415f8d0 2024-03-29T04:11:22.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.446+0000 7f5330ff9640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 0x7f532006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.446+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f532808b930 con 0x7f533415f8d0 2024-03-29T04:11:22.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.446+0000 7f5333fff640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 0x7f532006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.447+0000 7f5333fff640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 0x7f532006ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5334161bb0 tx=0x7f5324008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.451+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f53340749e0 con 0x7f533415f8d0 2024-03-29T04:11:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.656+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5334075890 con 0x7f533415f8d0 2024-03-29T04:11:22.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.657+0000 7f5330ff9640 1 -- 172.21.15.181:0/3841592718 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f533415cea0 con 0x7f533415f8d0 2024-03-29T04:11:22.636 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 msgr2=0x7f532006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 0x7f532006ea20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5334161bb0 tx=0x7f5324008040 comp rx=0 tx=0).stop 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 msgr2=0x7f533415fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f53280072c0 tx=0x7f5328007780 comp rx=0 tx=0).stop 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 shutdown_connections 2024-03-29T04:11:22.639 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53340ad940 0x7f53340addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f533415f8d0 0x7f533415fd50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f532006c560 0x7f532006ea20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 --2- 172.21.15.181:0/3841592718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f531c004d40 0x7f533415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 >> 172.21.15.181:0/3841592718 conn(0x7f53340678f0 msgr2=0x7f53340b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 shutdown_connections 2024-03-29T04:11:22.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.660+0000 7f533a396640 1 -- 172.21.15.181:0/3841592718 wait complete. 2024-03-29T04:11:22.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 Processor -- start 2024-03-29T04:11:22.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 -- start start 2024-03-29T04:11:22.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f800b5dc0 0x7f8f800b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f800b66e0 con 0x7f8f8006c620 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f8f800d2ff0 con 0x7f8f800b5dc0 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.783+0000 7f8f85baf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f800d32d0 con 0x7f8f800ba090 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f800b5dc0 0x7f8f800b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7ffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8f8006c620 0x7f8f8006cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36086/0 (socket says 172.21.15.181:36086) 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7ffff640 1 -- 172.21.15.181:0/1070389503 learned_addr learned my addr 172.21.15.181:0/1070389503 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7effd640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/1070389503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 msgr2=0x7f8f800d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7f7fe640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/1070389503 >> v1:172.21.15.181:6789/0 conn(0x7f8f8006c620 legacy=0x7f8f8006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.784+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/1070389503 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f800d35b0 con 0x7f8f800b5dc0 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.785+0000 7f8f7f7fe640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800b5dc0 0x7f8f800b61a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009d90 tx=0x7f8f7802bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac150a028236e9ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.785+0000 7f8f7e7fc640 1 -- 172.21.15.181:0/1070389503 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78039070 con 0x7f8f800b5dc0 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.785+0000 7f8f7e7fc640 1 -- 172.21.15.181:0/1070389503 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7802ca10 con 0x7f8f800b5dc0 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.785+0000 7f8f7e7fc640 1 -- 172.21.15.181:0/1070389503 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7802cde0 con 0x7f8f800b5dc0 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 -- 172.21.15.181:0/1070389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800b5dc0 msgr2=0x7f8f800b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800b5dc0 0x7f8f800b61a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009d90 tx=0x7f8f7802bce0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 -- 172.21.15.181:0/1070389503 shutdown_connections 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 --2- 172.21.15.181:0/1070389503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f800b5dc0 0x7f8f800b61a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 -- 172.21.15.181:0/1070389503 >> 172.21.15.181:0/1070389503 conn(0x7f8f800678f0 msgr2=0x7f8f800d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 -- 172.21.15.181:0/1070389503 shutdown_connections 2024-03-29T04:11:22.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.786+0000 7f8f85baf640 1 -- 172.21.15.181:0/1070389503 wait complete. 2024-03-29T04:11:22.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.787+0000 7f8f85baf640 1 Processor -- start 2024-03-29T04:11:22.768 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.787+0000 7f8f85baf640 1 -- start start 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800b5dc0 0x7f8f80154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f800d4090 con 0x7f8f8006c620 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f800d3b70 con 0x7f8f800b5dc0 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f85baf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f800b83e0 con 0x7f8f800ba090 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55964/0 (socket says 172.21.15.181:55964) 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/3243709364 learned_addr learned my addr 172.21.15.181:0/3243709364 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7ffff640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7effd640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800b5dc0 0x7f8f80154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 msgr2=0x7f8f800c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.769 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800b5dc0 msgr2=0x7f8f80154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800b5dc0 0x7f8f80154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 -- 172.21.15.181:0/3243709364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f80157b70 con 0x7f8f8006c620 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f7f7fe640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f78037040 tx=0x7f8f78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.788+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78044020 con 0x7f8f8006c620 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.789+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f78034b70 con 0x7f8f8006c620 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.789+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78039040 con 0x7f8f8006c620 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.789+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f800cca20 con 0x7f8f8006c620 2024-03-29T04:11:22.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.789+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f800ccfe0 con 0x7f8f8006c620 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.790+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f80075890 con 0x7f8f8006c620 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.790+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f78047020 con 0x7f8f8006c620 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.790+0000 7f8f77fff640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 0x7f8f6006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.791+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f8f780adca0 con 0x7f8f8006c620 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.791+0000 7f8f7effd640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 0x7f8f6006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:22.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.791+0000 7f8f7effd640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 0x7f8f6006ea20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ce410 tx=0x7f8f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:22.774 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.795+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8f7804e070 con 0x7f8f8006c620 2024-03-29T04:11:22.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.975+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f80075040 con 0x7f8f8006c620 2024-03-29T04:11:22.954 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.975+0000 7f8f77fff640 1 -- 172.21.15.181:0/3243709364 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f8f7807c110 con 0x7f8f8006c620 2024-03-29T04:11:22.955 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 msgr2=0x7f8f6006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 0x7f8f6006ea20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ce410 tx=0x7f8f68008040 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 msgr2=0x7f8f80157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8f78037040 tx=0x7f8f78004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 shutdown_connections 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f8f6006c560 0x7f8f6006ea20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f800ba090 0x7f8f800c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f800b5dc0 0x7f8f80154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 --2- 172.21.15.181:0/3243709364 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f8006c620 0x7f8f80157630 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 >> 172.21.15.181:0/3243709364 conn(0x7f8f800678f0 msgr2=0x7f8f800cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 shutdown_connections 2024-03-29T04:11:22.957 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:22.977+0000 7f8f85baf640 1 -- 172.21.15.181:0/3243709364 wait complete. 2024-03-29T04:11:22.971 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.097+0000 7fa8c3e4a640 1 Processor -- start 2024-03-29T04:11:23.076 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.097+0000 7fa8c3e4a640 1 -- start start 2024-03-29T04:11:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c3e4a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8bc0b5dc0 0x7fa8bc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c3e4a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa8bc0b73d0 0x7fa8bc0b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c3e4a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc0b66e0 con 0x7fa8bc0ba090 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c3e4a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0d2d90 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c3e4a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0d2ff0 con 0x7fa8bc0b73d0 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c13e5640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8bc0ba090 0x7fa8bc0d2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36090/0 (socket says 172.21.15.181:36090) 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c1be6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8bc0b5dc0 0x7fa8bc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c13e5640 1 -- 172.21.15.181:0/2276068776 learned_addr learned my addr 172.21.15.181:0/2276068776 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c23e7640 1 --2- 172.21.15.181:0/2276068776 >> v2:172.21.15.187:3300/0 conn(0x7fa8bc0b73d0 0x7fa8bc0b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c0be4640 1 -- 172.21.15.181:0/2276068776 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1395353439 0 0) 0x7fa8bc0b66e0 con 0x7fa8bc0ba090 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c0be4640 1 -- 172.21.15.181:0/2276068776 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a8003520 con 0x7fa8bc0ba090 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c1be6640 1 -- 172.21.15.181:0/2276068776 >> v2:172.21.15.187:3300/0 conn(0x7fa8bc0b73d0 msgr2=0x7fa8bc0b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c1be6640 1 --2- 172.21.15.181:0/2276068776 >> v2:172.21.15.187:3300/0 conn(0x7fa8bc0b73d0 0x7fa8bc0b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c1be6640 1 -- 172.21.15.181:0/2276068776 >> v1:172.21.15.181:6789/0 conn(0x7fa8bc0ba090 legacy=0x7fa8bc0d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.079 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.098+0000 7fa8c1be6640 1 -- 172.21.15.181:0/2276068776 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc0d32d0 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.099+0000 7fa8c1be6640 1 --2- 172.21.15.181:0/2276068776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0b5dc0 0x7fa8bc0b61a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0d5420 tx=0x7fa8b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7186c390c9088d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.099+0000 7fa8c0be4640 1 -- 172.21.15.181:0/2276068776 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035020 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.099+0000 7fa8c0be4640 1 -- 172.21.15.181:0/2276068776 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8031830 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.080 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.099+0000 7fa8c0be4640 1 -- 172.21.15.181:0/2276068776 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8031c00 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/2276068776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0b5dc0 msgr2=0x7fa8bc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/2276068776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0b5dc0 0x7fa8bc0b61a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0d5420 tx=0x7fa8b802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/2276068776 shutdown_connections 2024-03-29T04:11:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/2276068776 >> v2:172.21.15.187:3300/0 conn(0x7fa8bc0b73d0 0x7fa8bc0b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.084 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/2276068776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0b5dc0 0x7fa8bc0b61a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/2276068776 >> 172.21.15.181:0/2276068776 conn(0x7fa8bc0678f0 msgr2=0x7fa8bc0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/2276068776 shutdown_connections 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.100+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/2276068776 wait complete. 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.101+0000 7fa8c3e4a640 1 Processor -- start 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.101+0000 7fa8c3e4a640 1 -- start start 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8bc0b5dc0 0x7fa8bc14d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 0x7fa8bc1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d3b70 con 0x7fa8bc0ba090 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d3890 con 0x7fa8bc0b73d0 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c3e4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d4090 con 0x7fa8bc0b5dc0 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55984/0 (socket says 172.21.15.181:55984) 2024-03-29T04:11:23.085 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 -- 172.21.15.181:0/534853151 learned_addr learned my addr 172.21.15.181:0/534853151 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c1be6640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8bc0b5dc0 0x7fa8bc14d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c13e5640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 0x7fa8bc1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 -- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8bc0b5dc0 msgr2=0x7fa8bc14d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8bc0b5dc0 0x7fa8bc14d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 -- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 msgr2=0x7fa8bc1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 0x7fa8bc1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 -- 172.21.15.181:0/534853151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc14db40 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c13e5640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 0x7fa8bc1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8c23e7640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400efc0 tx=0x7fa8b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.102+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4018020 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.103+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b4014480 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.103+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc14de20 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.103+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4014870 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.103+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc14e3e0 con 0x7fa8bc0ba090 2024-03-29T04:11:23.086 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.103+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc160810 con 0x7fa8bc0ba090 2024-03-29T04:11:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.104+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa8b4005d10 con 0x7fa8bc0ba090 2024-03-29T04:11:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.104+0000 7fa8b27fc640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.105+0000 7fa8c1be6640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.105+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fa8b408c660 con 0x7fa8bc0ba090 2024-03-29T04:11:23.087 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.105+0000 7fa8c1be6640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 0x7fa8a006e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.088 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.109+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa8b4022030 con 0x7fa8bc0ba090 2024-03-29T04:11:23.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.289+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8bc0b7b80 con 0x7fa8bc0ba090 2024-03-29T04:11:23.269 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.290+0000 7fa8b27fc640 1 -- 172.21.15.181:0/534853151 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fa8b4010040 con 0x7fa8bc0ba090 2024-03-29T04:11:23.269 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:11:23.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 msgr2=0x7fa8a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.271 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 0x7fa8a006e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8008030 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 msgr2=0x7fa8bc14cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400efc0 tx=0x7fa8b400c490 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 shutdown_connections 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8bc0ba090 0x7fa8bc14cc20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc0b73d0 0x7fa8bc1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa8a006c530 0x7fa8a006e9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 --2- 172.21.15.181:0/534853151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8bc0b5dc0 0x7fa8bc14d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 >> 172.21.15.181:0/534853151 conn(0x7fa8bc0678f0 msgr2=0x7fa8bc0d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 shutdown_connections 2024-03-29T04:11:23.272 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.292+0000 7fa8c3e4a640 1 -- 172.21.15.181:0/534853151 wait complete. 2024-03-29T04:11:23.282 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:23.385 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.406+0000 7f8733e4a640 1 Processor -- start 2024-03-29T04:11:23.386 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.406+0000 7f8733e4a640 1 -- start start 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.407+0000 7f8733e4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.407+0000 7f8733e4a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f872c0d19c0 0x7f872c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.407+0000 7f8733e4a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f872c0d3120 con 0x7f872c0cec60 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.407+0000 7f8733e4a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f872c0cfc40 con 0x7f872c0d19c0 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.407+0000 7f8733e4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f872c0d3400 con 0x7f872c15e7a0 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.387 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f87323e7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f872c0d19c0 0x7f872c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:55458/0 (socket says 172.21.15.181:55458) 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 -- 172.21.15.181:0/1364144379 learned_addr learned my addr 172.21.15.181:0/1364144379 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 -- 172.21.15.181:0/1364144379 >> v2:172.21.15.187:3300/0 conn(0x7f872c0d19c0 msgr2=0x7f872c0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 --2- 172.21.15.181:0/1364144379 >> v2:172.21.15.187:3300/0 conn(0x7f872c0d19c0 0x7f872c0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 -- 172.21.15.181:0/1364144379 >> v1:172.21.15.187:6789/0 conn(0x7f872c0cec60 legacy=0x7f872c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.408+0000 7f8731be6640 1 -- 172.21.15.181:0/1364144379 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c0d36e0 con 0x7f872c15e7a0 2024-03-29T04:11:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.409+0000 7f8731be6640 1 --2- 172.21.15.181:0/1364144379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8728009870 tx=0x7f872802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b5b72f9744feac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.409+0000 7f8730be4640 1 -- 172.21.15.181:0/1364144379 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728035020 con 0x7f872c15e7a0 2024-03-29T04:11:23.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.409+0000 7f8730be4640 1 -- 172.21.15.181:0/1364144379 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728031640 con 0x7f872c15e7a0 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.409+0000 7f8730be4640 1 -- 172.21.15.181:0/1364144379 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872802d040 con 0x7f872c15e7a0 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 -- 172.21.15.181:0/1364144379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 msgr2=0x7f872c0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 --2- 172.21.15.181:0/1364144379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8728009870 tx=0x7f872802b8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 -- 172.21.15.181:0/1364144379 shutdown_connections 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 --2- 172.21.15.181:0/1364144379 >> v2:172.21.15.187:3300/0 conn(0x7f872c0d19c0 0x7f872c0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 --2- 172.21.15.181:0/1364144379 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c15e7a0 0x7f872c0cf3f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 -- 172.21.15.181:0/1364144379 >> 172.21.15.181:0/1364144379 conn(0x7f872c0676b0 msgr2=0x7f872c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 -- 172.21.15.181:0/1364144379 shutdown_connections 2024-03-29T04:11:23.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.410+0000 7f8733e4a640 1 -- 172.21.15.181:0/1364144379 wait complete. 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.411+0000 7f8733e4a640 1 Processor -- start 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.411+0000 7f8733e4a640 1 -- start start 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f872c0cec60 0x7f872c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c154be0 0x7f872c0b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f872c0d3ee0 con 0x7f872c0d19c0 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f872c0cf930 con 0x7f872c0cec60 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f872c0d4060 con 0x7f872c154be0 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:55998/0 (socket says 172.21.15.181:55998) 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 -- 172.21.15.181:0/3705700497 learned_addr learned my addr 172.21.15.181:0/3705700497 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:23.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8731be6640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f872c0cec60 0x7f872c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87323e7640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c154be0 0x7f872c0b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 -- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c154be0 msgr2=0x7f872c0b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c154be0 0x7f872c0b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 -- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f872c0cec60 msgr2=0x7f872c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f872c0cec60 0x7f872c0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 -- 172.21.15.181:0/3705700497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c0cc3b0 con 0x7f872c0d19c0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87313e5640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c3c0 tx=0x7f871c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c0052d0 con 0x7f872c0d19c0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c005490 con 0x7f872c0d19c0 2024-03-29T04:11:23.394 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.412+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c15eb90 con 0x7f872c0d19c0 2024-03-29T04:11:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.413+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c007260 con 0x7f872c0d19c0 2024-03-29T04:11:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.413+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c15f150 con 0x7f872c0d19c0 2024-03-29T04:11:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.414+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c075900 con 0x7f872c0d19c0 2024-03-29T04:11:23.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.414+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f871c007400 con 0x7f872c0d19c0 2024-03-29T04:11:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.414+0000 7f87227fc640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 0x7f871006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.415+0000 7f8731be6640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 0x7f871006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.415+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f871c08ac90 con 0x7f872c0d19c0 2024-03-29T04:11:23.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.415+0000 7f8731be6640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 0x7f871006e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8728036000 tx=0x7f872803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.398 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.419+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f871c058b40 con 0x7f872c0d19c0 2024-03-29T04:11:23.594 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.614+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f872c074e60 con 0x7f872c0d19c0 2024-03-29T04:11:23.594 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.615+0000 7f87227fc640 1 -- 172.21.15.181:0/3705700497 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f871c05ce40 con 0x7f872c0d19c0 2024-03-29T04:11:23.594 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:11:23.596 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 msgr2=0x7f871006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 0x7f871006e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8728036000 tx=0x7f872803c040 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 msgr2=0x7f872c157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c3c0 tx=0x7f871c00c880 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 shutdown_connections 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f871006c530 0x7f871006e9f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f872c154be0 0x7f872c0b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f872c0d19c0 0x7f872c157620 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 --2- 172.21.15.181:0/3705700497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f872c0cec60 0x7f872c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 >> 172.21.15.181:0/3705700497 conn(0x7f872c0676b0 msgr2=0x7f872c0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 shutdown_connections 2024-03-29T04:11:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.617+0000 7f8733e4a640 1 -- 172.21.15.181:0/3705700497 wait complete. 2024-03-29T04:11:23.607 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-29T04:11:23.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:23.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-29T04:11:23.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:11:23.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:23.663 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T04:11:23.695 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:11:23.695 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:11:23.695 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:23.712 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:11:23.712 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:23.769 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.890+0000 7fad16a22640 1 Processor -- start 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.890+0000 7fad16a22640 1 -- start start 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad16a22640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad1015e950 0x7fad100d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad16a22640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad100d36f0 con 0x7fad100d30d0 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad16a22640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad100d39d0 con 0x7fad100d48c0 2024-03-29T04:11:23.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad16a22640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fad100d3cb0 con 0x7fad1015e950 2024-03-29T04:11:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad0ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad1015e950 0x7fad100d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad14fbf640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fad100d30d0 0x7fad100b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36094/0 (socket says 172.21.15.181:36094) 2024-03-29T04:11:23.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad14fbf640 1 -- 172.21.15.181:0/2088119943 learned_addr learned my addr 172.21.15.181:0/2088119943 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad0ffff640 1 -- 172.21.15.181:0/2088119943 >> v1:172.21.15.187:6789/0 conn(0x7fad100d48c0 legacy=0x7fad100d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad0ffff640 1 -- 172.21.15.181:0/2088119943 >> v1:172.21.15.181:6789/0 conn(0x7fad100d30d0 legacy=0x7fad100b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.891+0000 7fad0ffff640 1 -- 172.21.15.181:0/2088119943 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad100b7ac0 con 0x7fad1015e950 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.892+0000 7fad0ffff640 1 --2- 172.21.15.181:0/2088119943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100d4380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fad000097a0 tx=0x7fad0002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59c89b9f9bcb4d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.892+0000 7fad0effd640 1 -- 172.21.15.181:0/2088119943 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0002f810 con 0x7fad1015e950 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.892+0000 7fad0effd640 1 -- 172.21.15.181:0/2088119943 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0002f9b0 con 0x7fad1015e950 2024-03-29T04:11:23.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.892+0000 7fad0effd640 1 -- 172.21.15.181:0/2088119943 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0002fd80 con 0x7fad1015e950 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 -- 172.21.15.181:0/2088119943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 msgr2=0x7fad100d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 --2- 172.21.15.181:0/2088119943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100d4380 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fad000097a0 tx=0x7fad0002b9f0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 -- 172.21.15.181:0/2088119943 shutdown_connections 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 --2- 172.21.15.181:0/2088119943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100d4380 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 -- 172.21.15.181:0/2088119943 >> 172.21.15.181:0/2088119943 conn(0x7fad100676e0 msgr2=0x7fad10067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 -- 172.21.15.181:0/2088119943 shutdown_connections 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.893+0000 7fad16a22640 1 -- 172.21.15.181:0/2088119943 wait complete. 2024-03-29T04:11:23.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.894+0000 7fad16a22640 1 Processor -- start 2024-03-29T04:11:23.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.894+0000 7fad16a22640 1 -- start start 2024-03-29T04:11:23.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad100d30d0 0x7fad1014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad100d48c0 0x7fad1015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad1015ed30 con 0x7fad1015e950 2024-03-29T04:11:23.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad100d4d00 con 0x7fad100d48c0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad16a22640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad1015eeb0 con 0x7fad100d30d0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56026/0 (socket says 172.21.15.181:56026) 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 -- 172.21.15.181:0/97868249 learned_addr learned my addr 172.21.15.181:0/97868249 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad0ffff640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad100d30d0 0x7fad1014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad0f7fe640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad100d48c0 0x7fad1015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 -- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad100d30d0 msgr2=0x7fad1014ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad100d30d0 0x7fad1014ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 -- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad100d48c0 msgr2=0x7fad1015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad100d48c0 0x7fad1015f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.895+0000 7fad14fbf640 1 -- 172.21.15.181:0/97868249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1014b050 con 0x7fad1015e950 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad14fbf640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad0800cbf0 tx=0x7fad0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08017070 con 0x7fad1015e950 2024-03-29T04:11:23.876 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0800ad40 con 0x7fad1015e950 2024-03-29T04:11:23.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1015fa10 con 0x7fad1015e950 2024-03-29T04:11:23.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08012600 con 0x7fad1015e950 2024-03-29T04:11:23.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.896+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1015ffd0 con 0x7fad1015e950 2024-03-29T04:11:23.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.897+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad10075a10 con 0x7fad1015e950 2024-03-29T04:11:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.898+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fad0801a070 con 0x7fad1015e950 2024-03-29T04:11:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.898+0000 7fad0cff9640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 0x7facfc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.898+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fad0808b480 con 0x7fad1015e950 2024-03-29T04:11:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.898+0000 7fad0ffff640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 0x7facfc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:23.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.898+0000 7fad0ffff640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 0x7facfc06e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad00036000 tx=0x7fad00008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:23.881 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:23.902+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fad08059330 con 0x7fad1015e950 2024-03-29T04:11:24.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.080+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad100b6e90 con 0x7fad1015e950 2024-03-29T04:11:24.061 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.081+0000 7fad0cff9640 1 -- 172.21.15.181:0/97868249 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fad0805d630 con 0x7fad1015e950 2024-03-29T04:11:24.061 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:24.061 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":41,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:58.390325+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.083+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 msgr2=0x7facfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.083+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 0x7facfc06e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad00036000 tx=0x7fad00008030 comp rx=0 tx=0).stop 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.083+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 msgr2=0x7fad100ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.083+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad0800cbf0 tx=0x7fad0800a430 comp rx=0 tx=0).stop 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 shutdown_connections 2024-03-29T04:11:24.063 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1015e950 0x7fad100ca170 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad100d48c0 0x7fad1015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7facfc06c530 0x7facfc06e9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 --2- 172.21.15.181:0/97868249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad100d30d0 0x7fad1014ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 >> 172.21.15.181:0/97868249 conn(0x7fad100676e0 msgr2=0x7fad100cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 shutdown_connections 2024-03-29T04:11:24.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.084+0000 7fad16a22640 1 -- 172.21.15.181:0/97868249 wait complete. 2024-03-29T04:11:24.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:10:20.863137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:10:21.635882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:11:24.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:24.178 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.199+0000 7f79facf1640 1 Processor -- start 2024-03-29T04:11:24.179 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.199+0000 7f79facf1640 1 -- start start 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79facf1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40b5dc0 0x7f79f40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79facf1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 0x7f79f40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79facf1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f40b66e0 con 0x7f79f406c620 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79facf1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f79f40d2ff0 con 0x7f79f40b5dc0 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79facf1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79f40d32d0 con 0x7f79f40ba090 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40b5dc0 0x7f79f40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f40b5dc0 0x7f79f40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56028/0 (socket says 172.21.15.181:56028) 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 -- 172.21.15.181:0/2143601618 learned_addr learned my addr 172.21.15.181:0/2143601618 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f3fff640 1 --2- 172.21.15.181:0/2143601618 >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 0x7f79f40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 -- 172.21.15.181:0/2143601618 >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 msgr2=0x7f79f40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/2143601618 >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 0x7f79f40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.200+0000 7f79f8a8d640 1 -- 172.21.15.181:0/2143601618 >> v1:172.21.15.187:6789/0 conn(0x7f79f406c620 legacy=0x7f79f406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.201+0000 7f79f8a8d640 1 -- 172.21.15.181:0/2143601618 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f40d35b0 con 0x7f79f40b5dc0 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.201+0000 7f79f3fff640 1 --2- 172.21.15.181:0/2143601618 >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 0x7f79f40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/2143601618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40b61a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b7240 tx=0x7f79e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b05e43a53873f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79f37fe640 1 -- 172.21.15.181:0/2143601618 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8035020 con 0x7f79f40b5dc0 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79f37fe640 1 -- 172.21.15.181:0/2143601618 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8031870 con 0x7f79f40b5dc0 2024-03-29T04:11:24.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79f37fe640 1 -- 172.21.15.181:0/2143601618 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8031c40 con 0x7f79f40b5dc0 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 -- 172.21.15.181:0/2143601618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 msgr2=0x7f79f40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 --2- 172.21.15.181:0/2143601618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40b61a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b7240 tx=0x7f79e802b910 comp rx=0 tx=0).stop 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 -- 172.21.15.181:0/2143601618 shutdown_connections 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 --2- 172.21.15.181:0/2143601618 >> v2:172.21.15.187:3300/0 conn(0x7f79f40ba090 0x7f79f40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 --2- 172.21.15.181:0/2143601618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40b61a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 -- 172.21.15.181:0/2143601618 >> 172.21.15.181:0/2143601618 conn(0x7f79f40678f0 msgr2=0x7f79f40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 -- 172.21.15.181:0/2143601618 shutdown_connections 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.202+0000 7f79facf1640 1 -- 172.21.15.181:0/2143601618 wait complete. 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 Processor -- start 2024-03-29T04:11:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 -- start start 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 0x7f79f4157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 0x7f79f4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40d4090 con 0x7f79f40b5dc0 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40d3b70 con 0x7f79f40ba090 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.204+0000 7f79facf1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79f40b83e0 con 0x7f79f406c620 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56030/0 (socket says 172.21.15.181:56030) 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 -- 172.21.15.181:0/3635802785 learned_addr learned my addr 172.21.15.181:0/3635802785 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 0x7f79f4157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f928e640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 0x7f79f4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 -- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 msgr2=0x7f79f4157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 0x7f79f4157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.186 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 -- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 msgr2=0x7f79f4160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 0x7f79f4160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 -- 172.21.15.181:0/3635802785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f4161490 con 0x7f79f40b5dc0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 0x7f79f4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f928e640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 0x7f79f4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f3fff640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c7f0 tx=0x7f79e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0015070 con 0x7f79f40b5dc0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e0007820 con 0x7f79f40b5dc0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f40ccc00 con 0x7f79f40b5dc0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007bd0 con 0x7f79f40b5dc0 2024-03-29T04:11:24.187 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.205+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f40cd1c0 con 0x7f79f40b5dc0 2024-03-29T04:11:24.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.207+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f79e0018070 con 0x7f79f40b5dc0 2024-03-29T04:11:24.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.207+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4075890 con 0x7f79f40b5dc0 2024-03-29T04:11:24.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.207+0000 7f79f17fa640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.207+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.207+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f79e008b910 con 0x7f79f40b5dc0 2024-03-29T04:11:24.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.208+0000 7f79f8a8d640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.211+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f79e0059e10 con 0x7f79f40b5dc0 2024-03-29T04:11:24.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.392+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79f40b9520 con 0x7f79f40b5dc0 2024-03-29T04:11:24.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.392+0000 7f79f17fa640 1 -- 172.21.15.181:0/3635802785 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f79e0059710 con 0x7f79f40b5dc0 2024-03-29T04:11:24.372 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:11:24.374 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 msgr2=0x7f79d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 msgr2=0x7f79f40cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c7f0 tx=0x7f79e000ccb0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 shutdown_connections 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f40ba090 0x7f79f4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f40b5dc0 0x7f79f40cc460 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f79d806c530 0x7f79d806e9f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 --2- 172.21.15.181:0/3635802785 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79f406c620 0x7f79f4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 >> 172.21.15.181:0/3635802785 conn(0x7f79f40678f0 msgr2=0x7f79f40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 shutdown_connections 2024-03-29T04:11:24.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.395+0000 7f79facf1640 1 -- 172.21.15.181:0/3635802785 wait complete. 2024-03-29T04:11:24.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:24.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.514+0000 7f55ece3c640 1 Processor -- start 2024-03-29T04:11:24.494 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.514+0000 7f55ece3c640 1 -- start start 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.515+0000 7f55ece3c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55e80ba090 0x7f55e80d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.515+0000 7f55ece3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 0x7f55e80b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.515+0000 7f55ece3c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e80d2ff0 con 0x7f55e80b5dc0 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.515+0000 7f55ece3c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f55e80d32d0 con 0x7f55e80ba090 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.515+0000 7f55ece3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55e80d35b0 con 0x7f55e80b73d0 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55e80ba090 0x7f55e80d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55e80ba090 0x7f55e80d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56046/0 (socket says 172.21.15.181:56046) 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 -- 172.21.15.181:0/742372589 learned_addr learned my addr 172.21.15.181:0/742372589 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e6d9d640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 0x7f55e80b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 -- 172.21.15.181:0/742372589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 msgr2=0x7f55e80b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 0x7f55e80b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 -- 172.21.15.181:0/742372589 >> v1:172.21.15.187:6789/0 conn(0x7f55e80b5dc0 legacy=0x7f55e80b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e5d9b640 1 -- 172.21.15.181:0/742372589 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e80d3890 con 0x7f55e80ba090 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.516+0000 7f55e6d9d640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 0x7f55e80b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55e5d9b640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80d2680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0098a0 tx=0x7f55dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770eec7f0d571c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55e559a640 1 -- 172.21.15.181:0/742372589 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035020 con 0x7f55e80ba090 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55e559a640 1 -- 172.21.15.181:0/742372589 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc031640 con 0x7f55e80ba090 2024-03-29T04:11:24.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55e559a640 1 -- 172.21.15.181:0/742372589 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc02d040 con 0x7f55e80ba090 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 -- 172.21.15.181:0/742372589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 msgr2=0x7f55e80d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80d2680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0098a0 tx=0x7f55dc02b8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 -- 172.21.15.181:0/742372589 shutdown_connections 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b73d0 0x7f55e80b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 --2- 172.21.15.181:0/742372589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80d2680 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 -- 172.21.15.181:0/742372589 >> 172.21.15.181:0/742372589 conn(0x7f55e80678f0 msgr2=0x7f55e80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 -- 172.21.15.181:0/742372589 shutdown_connections 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.517+0000 7f55ece3c640 1 -- 172.21.15.181:0/742372589 wait complete. 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 Processor -- start 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 -- start start 2024-03-29T04:11:24.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 0x7f55e80b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 0x7f55e815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55e815e880 con 0x7f55e80ba090 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55e80d4090 con 0x7f55e80b73d0 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55ece3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55e815ea00 con 0x7f55e80b5dc0 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.519+0000 7f55e6d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56054/0 (socket says 172.21.15.181:56054) 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 -- 172.21.15.181:0/3220491206 learned_addr learned my addr 172.21.15.181:0/3220491206 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e659c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 0x7f55e80b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e5d9b640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 0x7f55e815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 -- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 msgr2=0x7f55e80b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.501 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 0x7f55e80b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 -- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 msgr2=0x7f55e815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 0x7f55e815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 -- 172.21.15.181:0/3220491206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e8160050 con 0x7f55e80ba090 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e659c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 0x7f55e80b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e5d9b640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 0x7f55e815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:24.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55e6d9d640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55d800cef0 tx=0x7f55d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d801a070 con 0x7f55e80ba090 2024-03-29T04:11:24.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d800d860 con 0x7f55e80ba090 2024-03-29T04:11:24.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e8160330 con 0x7f55e80ba090 2024-03-29T04:11:24.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d800dc50 con 0x7f55e80ba090 2024-03-29T04:11:24.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.520+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e81608f0 con 0x7f55e80ba090 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.521+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e8075890 con 0x7f55e80ba090 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.522+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f55d801d070 con 0x7f55e80ba090 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.522+0000 7f55d6ffd640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 0x7f55c806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.522+0000 7f55e659c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 0x7f55c806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.522+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f55d808c0a0 con 0x7f55e80ba090 2024-03-29T04:11:24.504 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.523+0000 7f55e659c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 0x7f55c806ea20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f55e80b8b80 tx=0x7f55e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.505 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.526+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f55d805a660 con 0x7f55e80ba090 2024-03-29T04:11:24.686 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.707+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55e80cab70 con 0x7f55e80ba090 2024-03-29T04:11:24.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.707+0000 7f55d6ffd640 1 -- 172.21.15.181:0/3220491206 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f55d8059f60 con 0x7f55e80ba090 2024-03-29T04:11:24.687 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 msgr2=0x7f55c806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 0x7f55c806ea20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f55e80b8b80 tx=0x7f55e0009040 comp rx=0 tx=0).stop 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 msgr2=0x7f55e80cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f55d800cef0 tx=0x7f55d800a710 comp rx=0 tx=0).stop 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 shutdown_connections 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55e80ba090 0x7f55e80cb410 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55e80b73d0 0x7f55e815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.689 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f55c806c560 0x7f55c806ea20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 --2- 172.21.15.181:0/3220491206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55e80b5dc0 0x7f55e80b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 >> 172.21.15.181:0/3220491206 conn(0x7f55e80678f0 msgr2=0x7f55e80b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 shutdown_connections 2024-03-29T04:11:24.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.709+0000 7f55ece3c640 1 -- 172.21.15.181:0/3220491206 wait complete. 2024-03-29T04:11:24.703 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:24.842 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.863+0000 7f4ebc7ef640 1 Processor -- start 2024-03-29T04:11:24.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.863+0000 7f4ebc7ef640 1 -- start start 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4ebc7ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4ebc7ef640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 0x7f4eb40d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4ebc7ef640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb40b66e0 con 0x7f4eb406c620 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4ebc7ef640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb40d2ff0 con 0x7f4eb40ba090 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4ebc7ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb40d32d0 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4eba58b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.864+0000 7f4eb9d8a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 0x7f4eb40d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:55494/0 (socket says 172.21.15.181:55494) 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 -- 172.21.15.181:0/3282850787 learned_addr learned my addr 172.21.15.181:0/3282850787 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4ebad8c640 1 --1- 172.21.15.181:0/3282850787 >> v1:172.21.15.187:6789/0 conn(0x7f4eb406c620 0x7f4eb406cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:40640/0 (socket says 172.21.15.181:40640) 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 -- 172.21.15.181:0/3282850787 >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 msgr2=0x7f4eb40d2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 --2- 172.21.15.181:0/3282850787 >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 0x7f4eb40d2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 -- 172.21.15.181:0/3282850787 >> v1:172.21.15.187:6789/0 conn(0x7f4eb406c620 legacy=0x7f4eb406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eba58b640 1 -- 172.21.15.181:0/3282850787 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb40d35b0 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.865+0000 7f4eb9d8a640 1 --2- 172.21.15.181:0/3282850787 >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 0x7f4eb40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:24.848 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4eba58b640 1 --2- 172.21.15.181:0/3282850787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009e50 tx=0x7f4eac02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f35cb2de48575b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4eb9589640 1 -- 172.21.15.181:0/3282850787 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac02cb00 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4eb9589640 1 -- 172.21.15.181:0/3282850787 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac02cca0 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4eb9589640 1 -- 172.21.15.181:0/3282850787 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac034610 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/3282850787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 msgr2=0x7f4eb40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/3282850787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009e50 tx=0x7f4eac02bf60 comp rx=0 tx=0).stop 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/3282850787 shutdown_connections 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/3282850787 >> v2:172.21.15.187:3300/0 conn(0x7f4eb40ba090 0x7f4eb40d2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/3282850787 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb40b61a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/3282850787 >> 172.21.15.181:0/3282850787 conn(0x7f4eb40678f0 msgr2=0x7f4eb40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/3282850787 shutdown_connections 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.866+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/3282850787 wait complete. 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 Processor -- start 2024-03-29T04:11:24.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 -- start start 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4eb406c620 0x7f4eb4155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb4161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb40d4090 con 0x7f4eb40ba090 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb40d3b70 con 0x7f4eb406c620 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebc7ef640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb40b83e0 con 0x7f4eb40b5dc0 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.868+0000 7f4ebad8c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56066/0 (socket says 172.21.15.181:56066) 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 -- 172.21.15.181:0/2926026855 learned_addr learned my addr 172.21.15.181:0/2926026855 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4eb9d8a640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb4161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4eba58b640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4eb406c620 0x7f4eb4155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 -- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 msgr2=0x7f4eb4161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb4161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 -- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4eb406c620 msgr2=0x7f4eb4155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:24.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4eb406c620 0x7f4eb4155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 -- 172.21.15.181:0/2926026855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb41549e0 con 0x7f4eb40ba090 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebad8c640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c430 tx=0x7f4ea400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40074e0 con 0x7f4eb40ba090 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea4007680 con 0x7f4eb40ba090 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb4154cc0 con 0x7f4eb40ba090 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40049e0 con 0x7f4eb40ba090 2024-03-29T04:11:24.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.869+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb415f8b0 con 0x7f4eb40ba090 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.870+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb40b9230 con 0x7f4eb40ba090 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.871+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4ea401a020 con 0x7f4eb40ba090 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.871+0000 7f4eb2ffd640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 0x7f4ea006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.871+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f4ea408af60 con 0x7f4eb40ba090 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.871+0000 7f4eba58b640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 0x7f4ea006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:24.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.871+0000 7f4eba58b640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 0x7f4ea006e9f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4eac035000 tx=0x7f4eac030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:24.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:24.880+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4eb40b9230 con 0x7f4eb40ba090 2024-03-29T04:11:25.198 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.219+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4eb415fab0 con 0x7f4eb40ba090 2024-03-29T04:11:25.199 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.219+0000 7f4eb2ffd640 1 -- 172.21.15.181:0/2926026855 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f4eb415fab0 con 0x7f4eb40ba090 2024-03-29T04:11:25.199 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 msgr2=0x7f4ea006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 0x7f4ea006e9f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4eac035000 tx=0x7f4eac030000 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 msgr2=0x7f4eb4156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400c430 tx=0x7f4ea400c8f0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 shutdown_connections 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4eb40ba090 0x7f4eb4156b70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f4ea006c530 0x7f4ea006e9f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4eb40b5dc0 0x7f4eb4161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 --2- 172.21.15.181:0/2926026855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4eb406c620 0x7f4eb4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.201 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 >> 172.21.15.181:0/2926026855 conn(0x7f4eb40678f0 msgr2=0x7f4eb40b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:25.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 shutdown_connections 2024-03-29T04:11:25.202 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.221+0000 7f4ebc7ef640 1 -- 172.21.15.181:0/2926026855 wait complete. 2024-03-29T04:11:25.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T04:11:25.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.339+0000 7f391e79d640 1 Processor -- start 2024-03-29T04:11:25.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.339+0000 7f391e79d640 1 -- start start 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39180cb050 0x7f39180cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f391815f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f391815f6c0 0x7f3918151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f391815c480 con 0x7f391815f6c0 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f39180cc140 con 0x7f39180cb050 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391e79d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f391815feb0 con 0x7f39180cbd00 2024-03-29T04:11:25.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f391815f6c0 0x7f3918151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f391815f6c0 0x7f3918151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56080/0 (socket says 172.21.15.181:56080) 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 -- 172.21.15.181:0/4094548167 learned_addr learned my addr 172.21.15.181:0/4094548167 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f3917fff640 1 --2- 172.21.15.181:0/4094548167 >> v2:172.21.15.187:3300/0 conn(0x7f39180cb050 0x7f39180cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f39177fe640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f391815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 -- 172.21.15.181:0/4094548167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 msgr2=0x7f391815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f391815f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 -- 172.21.15.181:0/4094548167 >> v2:172.21.15.187:3300/0 conn(0x7f39180cb050 msgr2=0x7f39180cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 --2- 172.21.15.181:0/4094548167 >> v2:172.21.15.187:3300/0 conn(0x7f39180cb050 0x7f39180cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f391cd3a640 1 -- 172.21.15.181:0/4094548167 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391815c910 con 0x7f391815f6c0 2024-03-29T04:11:25.321 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.340+0000 7f39177fe640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f391815f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391cd3a640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f3918151c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f391806e1a0 tx=0x7f391002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f61deeea7612d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f3916ffd640 1 -- 172.21.15.181:0/4094548167 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391002ea80 con 0x7f391815f6c0 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f3916ffd640 1 -- 172.21.15.181:0/4094548167 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391002ec20 con 0x7f391815f6c0 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f3916ffd640 1 -- 172.21.15.181:0/4094548167 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910009600 con 0x7f391815f6c0 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 -- 172.21.15.181:0/4094548167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 msgr2=0x7f3918151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f3918151c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f391806e1a0 tx=0x7f391002e0e0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 -- 172.21.15.181:0/4094548167 shutdown_connections 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f3918151c20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 --2- 172.21.15.181:0/4094548167 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f391815f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 --2- 172.21.15.181:0/4094548167 >> v2:172.21.15.187:3300/0 conn(0x7f39180cb050 0x7f39180cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 -- 172.21.15.181:0/4094548167 >> 172.21.15.181:0/4094548167 conn(0x7f391806d610 msgr2=0x7f391806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:25.323 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 -- 172.21.15.181:0/4094548167 shutdown_connections 2024-03-29T04:11:25.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.342+0000 7f391e79d640 1 -- 172.21.15.181:0/4094548167 wait complete. 2024-03-29T04:11:25.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.343+0000 7f391e79d640 1 Processor -- start 2024-03-29T04:11:25.324 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 -- start start 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39180cb050 0x7f39180c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f39181556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f391815c110 con 0x7f391815f6c0 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f391815be30 con 0x7f39180cb050 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391e79d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f391815fb50 con 0x7f39180cbd00 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391cd3a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391cd3a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56084/0 (socket says 172.21.15.181:56084) 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.344+0000 7f391cd3a640 1 -- 172.21.15.181:0/1629790481 learned_addr learned my addr 172.21.15.181:0/1629790481 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f39177fe640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f39181556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f3917fff640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39180cb050 0x7f39180c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 -- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 msgr2=0x7f39181556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.326 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f39181556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 -- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39180cb050 msgr2=0x7f39180c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39180cb050 0x7f39180c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 -- 172.21.15.181:0/1629790481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180ce960 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391cd3a640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3910038040 tx=0x7f3910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910045020 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910040660 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39181f3ab0 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391002ea80 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.345+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39180ce630 con 0x7f391815f6c0 2024-03-29T04:11:25.327 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.346+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39180cb430 con 0x7f391815f6c0 2024-03-29T04:11:25.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.347+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3910048020 con 0x7f391815f6c0 2024-03-29T04:11:25.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.348+0000 7f3914ff9640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.348+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f39100b0350 con 0x7f391815f6c0 2024-03-29T04:11:25.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.348+0000 7f3917fff640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.328 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.348+0000 7f3917fff640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 0x7f390406e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f390c006fd0 tx=0x7f390c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.331 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.352+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f39180cb430 con 0x7f391815f6c0 2024-03-29T04:11:25.524 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.544+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3918075900 con 0x7f391815f6c0 2024-03-29T04:11:25.524 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.545+0000 7f3914ff9640 1 -- 172.21.15.181:0/1629790481 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7f391004f030 con 0x7f391815f6c0 2024-03-29T04:11:25.524 INFO:teuthology.orchestra.run.smithi181.stderr:listed 0 entries 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 msgr2=0x7f390406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 0x7f390406e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f390c006fd0 tx=0x7f390c008040 comp rx=0 tx=0).stop 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 msgr2=0x7f391814f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3910038040 tx=0x7f3910004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 shutdown_connections 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f391815f6c0 0x7f391814f600 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f390406c530 0x7f390406e9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.527 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39180cbd00 0x7f39181556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 --2- 172.21.15.181:0/1629790481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39180cb050 0x7f39180c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 >> 172.21.15.181:0/1629790481 conn(0x7f391806d610 msgr2=0x7f39181596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:25.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 shutdown_connections 2024-03-29T04:11:25.528 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.547+0000 7f391e79d640 1 -- 172.21.15.181:0/1629790481 wait complete. 2024-03-29T04:11:25.537 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-29T04:11:25.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:25.537 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-29T04:11:25.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:11:25.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:25.573 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T04:11:25.631 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:11:25.631 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:11:25.631 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:25.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056349,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15338 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:11:25.647 INFO:tasks.cephfs.mount:Terminating background process 2024-03-29T04:11:25.647 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:11:25.647 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:25.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2267 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:11:25.689 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.811+0000 7f98d5e5b640 1 Processor -- start 2024-03-29T04:11:25.791 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.811+0000 7f98d5e5b640 1 -- start start 2024-03-29T04:11:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98d5e5b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98d015e950 0x7f98d00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98d5e5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d48c0 0x7f98d00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98d5e5b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d00d36f0 con 0x7f98d00d2f00 2024-03-29T04:11:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98d5e5b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f98d00d3cb0 con 0x7f98d015e950 2024-03-29T04:11:25.792 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98d5e5b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98d00d4d00 con 0x7f98d00d48c0 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98cf7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98d015e950 0x7f98d00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98cffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f98d00d2f00 0x7f98d00cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:36110/0 (socket says 172.21.15.181:36110) 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.812+0000 7f98cffff640 1 -- 172.21.15.181:0/1067196881 learned_addr learned my addr 172.21.15.181:0/1067196881 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98ceffd640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d48c0 0x7f98d00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98cf7fe640 1 -- 172.21.15.181:0/1067196881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d48c0 msgr2=0x7f98d00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d48c0 0x7f98d00d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98cf7fe640 1 -- 172.21.15.181:0/1067196881 >> v1:172.21.15.181:6789/0 conn(0x7f98d00d2f00 legacy=0x7f98d00cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98cf7fe640 1 -- 172.21.15.181:0/1067196881 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d00d39d0 con 0x7f98d015e950 2024-03-29T04:11:25.793 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98ce7fc640 1 -- 172.21.15.181:0/1067196881 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726869823 0 0) 0x7f98d00d36f0 con 0x7f98d00d2f00 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.813+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d015e950 0x7f98d00d4380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097a0 tx=0x7f98c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76656ffa23ff06 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98ce7fc640 1 -- 172.21.15.181:0/1067196881 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0034070 con 0x7f98d015e950 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98ce7fc640 1 -- 172.21.15.181:0/1067196881 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c0030610 con 0x7f98d015e950 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98ce7fc640 1 -- 172.21.15.181:0/1067196881 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00309e0 con 0x7f98d015e950 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 -- 172.21.15.181:0/1067196881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d015e950 msgr2=0x7f98d00d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d015e950 0x7f98d00d4380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097a0 tx=0x7f98c002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 -- 172.21.15.181:0/1067196881 shutdown_connections 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d48c0 0x7f98d00d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/1067196881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d015e950 0x7f98d00d4380 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 -- 172.21.15.181:0/1067196881 >> 172.21.15.181:0/1067196881 conn(0x7f98d00676e0 msgr2=0x7f98d0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 -- 172.21.15.181:0/1067196881 shutdown_connections 2024-03-29T04:11:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.814+0000 7f98d5e5b640 1 -- 172.21.15.181:0/1067196881 wait complete. 2024-03-29T04:11:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.815+0000 7f98d5e5b640 1 Processor -- start 2024-03-29T04:11:25.796 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 -- start start 2024-03-29T04:11:25.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 0x7f98d00caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98d015e950 0x7f98d0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00d3390 con 0x7f98d00d48c0 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00b7da0 con 0x7f98d015e950 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98d5e5b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98d015ed30 con 0x7f98d00d2f00 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56108/0 (socket says 172.21.15.181:56108) 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.816+0000 7f98ceffd640 1 -- 172.21.15.181:0/3392825742 learned_addr learned my addr 172.21.15.181:0/3392825742 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:25.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 0x7f98d00caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98cffff640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98d015e950 0x7f98d0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 -- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 msgr2=0x7f98d00caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 0x7f98d00caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 -- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98d015e950 msgr2=0x7f98d0156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98d015e950 0x7f98d0156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 -- 172.21.15.181:0/3392825742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d00caf80 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 0x7f98d00caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98ceffd640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e9b0 tx=0x7f98c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4017070 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c400c9a0 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d0160d10 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400cd90 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.817+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01612d0 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.818+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d0075a10 con 0x7f98d00d48c0 2024-03-29T04:11:25.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.819+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f98c401a070 con 0x7f98d00d48c0 2024-03-29T04:11:25.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.819+0000 7f98b7fff640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 0x7f98bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:25.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.819+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f98c408b810 con 0x7f98d00d48c0 2024-03-29T04:11:25.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.819+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 0x7f98bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:25.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.820+0000 7f98cf7fe640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 0x7f98bc06e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:25.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:25.824+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f98c4059d10 con 0x7f98d00d48c0 2024-03-29T04:11:26.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.042+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98d00cfdf0 con 0x7f98d00d48c0 2024-03-29T04:11:26.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.042+0000 7f98b7fff640 1 -- 172.21.15.181:0/3392825742 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f98c4059610 con 0x7f98d00d48c0 2024-03-29T04:11:26.022 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:26.022 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:44.376045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:11:26.022 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 18 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 msgr2=0x7f98bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 0x7f98bc06e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c003b040 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 msgr2=0x7f98d0160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e9b0 tx=0x7f98c400ee70 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 shutdown_connections 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98d015e950 0x7f98d0156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d00d48c0 0x7f98d0160330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f98bc06c530 0x7f98bc06e9f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 --2- 172.21.15.181:0/3392825742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98d00d2f00 0x7f98d00caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 >> 172.21.15.181:0/3392825742 conn(0x7f98d00676e0 msgr2=0x7f98d00cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 shutdown_connections 2024-03-29T04:11:26.024 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.044+0000 7f98d5e5b640 1 -- 172.21.15.181:0/3392825742 wait complete. 2024-03-29T04:11:26.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:26.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.162+0000 7ffa12b50640 1 Processor -- start 2024-03-29T04:11:26.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.162+0000 7ffa12b50640 1 -- start start 2024-03-29T04:11:26.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffa0c15e950 0x7ffa0c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d48c0 0x7ffa0c0d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c0d3cb0 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffa0c0d4d00 con 0x7ffa0c15e950 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa12b50640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d36f0 con 0x7ffa0c0d48c0 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56112/0 (socket says 172.21.15.181:56112) 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 -- 172.21.15.181:0/2215326685 learned_addr learned my addr 172.21.15.181:0/2215326685 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa108ec640 1 --2- 172.21.15.181:0/2215326685 >> v2:172.21.15.187:3300/0 conn(0x7ffa0c15e950 0x7ffa0c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 -- 172.21.15.181:0/2215326685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d48c0 msgr2=0x7ffa0c0d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa0bfff640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d48c0 0x7ffa0c0d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d48c0 0x7ffa0c0d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 -- 172.21.15.181:0/2215326685 >> v2:172.21.15.187:3300/0 conn(0x7ffa0c15e950 msgr2=0x7ffa0c0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2215326685 >> v2:172.21.15.187:3300/0 conn(0x7ffa0c15e950 0x7ffa0c0d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.163+0000 7ffa110ed640 1 -- 172.21.15.181:0/2215326685 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0d39d0 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0d5510 tx=0x7ffa0402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9316b02420b3b8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa0b7fe640 1 -- 172.21.15.181:0/2215326685 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0402cb00 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa0b7fe640 1 -- 172.21.15.181:0/2215326685 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0402cca0 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa0b7fe640 1 -- 172.21.15.181:0/2215326685 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04034610 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa12b50640 1 -- 172.21.15.181:0/2215326685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0d2f00 msgr2=0x7ffa0c0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.164+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0d5510 tx=0x7ffa0402bf60 comp rx=0 tx=0).stop 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 -- 172.21.15.181:0/2215326685 shutdown_connections 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0d2f00 0x7ffa0c0cef90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2215326685 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d48c0 0x7ffa0c0d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2215326685 >> v2:172.21.15.187:3300/0 conn(0x7ffa0c15e950 0x7ffa0c0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 -- 172.21.15.181:0/2215326685 >> 172.21.15.181:0/2215326685 conn(0x7ffa0c0676e0 msgr2=0x7ffa0c067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 -- 172.21.15.181:0/2215326685 shutdown_connections 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.165+0000 7ffa12b50640 1 -- 172.21.15.181:0/2215326685 wait complete. 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.166+0000 7ffa12b50640 1 Processor -- start 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.166+0000 7ffa12b50640 1 -- start start 2024-03-29T04:11:26.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 0x7ffa0c14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 0x7ffa0c1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0d3390 con 0x7ffa0c15e950 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c0b7da0 con 0x7ffa0c0d48c0 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c15ed30 con 0x7ffa0c0d2f00 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56126/0 (socket says 172.21.15.181:56126) 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 -- 172.21.15.181:0/2584578516 learned_addr learned my addr 172.21.15.181:0/2584578516 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa108ec640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 0x7ffa0c14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa0bfff640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 0x7ffa0c1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 -- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 msgr2=0x7ffa0c14a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 0x7ffa0c14a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 -- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 msgr2=0x7ffa0c1600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 0x7ffa0c1600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 -- 172.21.15.181:0/2584578516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c0ca7a0 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa108ec640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 0x7ffa0c14a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa0bfff640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 0x7ffa0c1600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa110ed640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009f40 tx=0x7ffa04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04043020 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0402ce50 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c0caa50 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04009510 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.167+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c14af50 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.168+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c075a10 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.169+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffa04046020 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.169+0000 7ffa097fa640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 0x7ff9f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.169+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7ffa040ae350 con 0x7ffa0c15e950 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.169+0000 7ffa108ec640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 0x7ff9f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.170+0000 7ffa108ec640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 0x7ff9f806e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b840 tx=0x7ff9fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.178+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ffa0404d020 con 0x7ffa0c15e950 2024-03-29T04:11:26.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.361+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa0c0cdcb0 con 0x7ffa0c15e950 2024-03-29T04:11:26.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.363+0000 7ffa097fa640 1 -- 172.21.15.181:0/2584578516 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7ffa0407c860 con 0x7ffa0c15e950 2024-03-29T04:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":41,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:58.390325+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:26.344 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 msgr2=0x7ff9f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 0x7ff9f806e9f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b840 tx=0x7ff9fc008040 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 msgr2=0x7ffa0c1605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009f40 tx=0x7ffa04004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 shutdown_connections 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c15e950 0x7ffa0c1605f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa0c0d48c0 0x7ffa0c1600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff9f806c530 0x7ff9f806e9f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 --2- 172.21.15.181:0/2584578516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa0c0d2f00 0x7ffa0c14a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 >> 172.21.15.181:0/2584578516 conn(0x7ffa0c0676e0 msgr2=0x7ffa0c0cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 shutdown_connections 2024-03-29T04:11:26.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.365+0000 7ffa12b50640 1 -- 172.21.15.181:0/2584578516 wait complete. 2024-03-29T04:11:26.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T04:09:03.587081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T04:10:20.863137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T04:10:21.635882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T04:11:26.357 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T04:11:26.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.486+0000 7ff26df15640 1 Processor -- start 2024-03-29T04:11:26.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.486+0000 7ff26df15640 1 -- start start 2024-03-29T04:11:26.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff268144cb0 0x7ff2680b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff26815f800 0x7ff268072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff268071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7ff2680b3e80 con 0x7ff26815f800 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff2680b35e0 con 0x7ff268144cb0 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff26df15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2680b38c0 con 0x7ff268072b80 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff26815f800 0x7ff268072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff26815f800 0x7ff268072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56132/0 (socket says 172.21.15.181:56132) 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 -- 172.21.15.181:0/2346511823 learned_addr learned my addr 172.21.15.181:0/2346511823 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff267fff640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff268071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff2677fe640 1 --2- 172.21.15.181:0/2346511823 >> v2:172.21.15.187:3300/0 conn(0x7ff268144cb0 0x7ff2680b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 -- 172.21.15.181:0/2346511823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 msgr2=0x7ff268071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff268071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 -- 172.21.15.181:0/2346511823 >> v2:172.21.15.187:3300/0 conn(0x7ff268144cb0 msgr2=0x7ff2680b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 --2- 172.21.15.181:0/2346511823 >> v2:172.21.15.187:3300/0 conn(0x7ff268144cb0 0x7ff2680b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.487+0000 7ff266ffd640 1 -- 172.21.15.181:0/2346511823 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680b3ba0 con 0x7ff26815f800 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.488+0000 7ff266ffd640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff268072640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff26806e3e0 tx=0x7ff25c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c0a5803c640778 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.488+0000 7ff2667fc640 1 -- 172.21.15.181:0/2346511823 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c035020 con 0x7ff26815f800 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.488+0000 7ff2667fc640 1 -- 172.21.15.181:0/2346511823 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c02d040 con 0x7ff26815f800 2024-03-29T04:11:26.469 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.488+0000 7ff2667fc640 1 -- 172.21.15.181:0/2346511823 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c031a80 con 0x7ff26815f800 2024-03-29T04:11:26.471 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 -- 172.21.15.181:0/2346511823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 msgr2=0x7ff268072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff268072640 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff26806e3e0 tx=0x7ff25c02b900 comp rx=0 tx=0).stop 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 -- 172.21.15.181:0/2346511823 shutdown_connections 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff268071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 --2- 172.21.15.181:0/2346511823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff268072640 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 --2- 172.21.15.181:0/2346511823 >> v2:172.21.15.187:3300/0 conn(0x7ff268144cb0 0x7ff2680b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 -- 172.21.15.181:0/2346511823 >> 172.21.15.181:0/2346511823 conn(0x7ff2680b5b00 msgr2=0x7ff26806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 -- 172.21.15.181:0/2346511823 shutdown_connections 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.489+0000 7ff26df15640 1 -- 172.21.15.181:0/2346511823 wait complete. 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.490+0000 7ff26df15640 1 Processor -- start 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 -- start start 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff2680acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff268144cb0 0x7ff2681514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.472 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff268071960 con 0x7ff26815f800 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff26815fc40 con 0x7ff268144cb0 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff26df15640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff268071ae0 con 0x7ff268072b80 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff267fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff267fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56142/0 (socket says 172.21.15.181:56142) 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.491+0000 7ff267fff640 1 -- 172.21.15.181:0/2646913404 learned_addr learned my addr 172.21.15.181:0/2646913404 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 -- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 msgr2=0x7ff2680acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff2680acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 -- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff268144cb0 msgr2=0x7ff2681514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff268144cb0 0x7ff2681514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 -- 172.21.15.181:0/2646913404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2680a9920 con 0x7ff26815f800 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff267fff640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff260007e40 tx=0x7ff26000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260014620 con 0x7ff26815f800 2024-03-29T04:11:26.473 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff260019070 con 0x7ff26815f800 2024-03-29T04:11:26.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2680a9c00 con 0x7ff26815f800 2024-03-29T04:11:26.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260014a20 con 0x7ff26815f800 2024-03-29T04:11:26.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.492+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2680aa1c0 con 0x7ff26815f800 2024-03-29T04:11:26.474 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.493+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff268075890 con 0x7ff26815f800 2024-03-29T04:11:26.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.493+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff260014bc0 con 0x7ff26815f800 2024-03-29T04:11:26.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.494+0000 7ff24ffff640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 0x7ff25406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.494+0000 7ff2677fe640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 0x7ff25406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.494+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7ff26008c230 con 0x7ff26815f800 2024-03-29T04:11:26.475 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.494+0000 7ff2677fe640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 0x7ff25406e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff258008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.502+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff26005a6a0 con 0x7ff26815f800 2024-03-29T04:11:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.733+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff2680b4030 con 0x7ff26815f800 2024-03-29T04:11:26.712 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.733+0000 7ff24ffff640 1 -- 172.21.15.181:0/2646913404 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7ff260059fa0 con 0x7ff26815f800 2024-03-29T04:11:26.713 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 1 2024-03-29T04:11:26.715 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 msgr2=0x7ff25406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 0x7ff25406e9f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff258008040 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 msgr2=0x7ff2681519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff260007e40 tx=0x7ff26000e4b0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 shutdown_connections 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff26815f800 0x7ff2681519e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff268144cb0 0x7ff2681514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff25406c530 0x7ff25406e9f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 --2- 172.21.15.181:0/2646913404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff268072b80 0x7ff2680acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 >> 172.21.15.181:0/2646913404 conn(0x7ff2680b5b00 msgr2=0x7ff2680afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 shutdown_connections 2024-03-29T04:11:26.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.736+0000 7ff26df15640 1 -- 172.21.15.181:0/2646913404 wait complete. 2024-03-29T04:11:26.726 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T04:11:26.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.852+0000 7f622e982640 1 Processor -- start 2024-03-29T04:11:26.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.852+0000 7f622e982640 1 -- start start 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622e982640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622e982640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 0x7f622815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622e982640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622815c990 con 0x7f62280b7950 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622e982640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6228071f50 con 0x7f6228071aa0 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622e982640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62280ccb70 con 0x7f62280d1980 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f62277fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f622cf1f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 0x7f622815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f62277fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:55562/0 (socket says 172.21.15.181:55562) 2024-03-29T04:11:26.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f62277fe640 1 -- 172.21.15.181:0/3111322643 learned_addr learned my addr 172.21.15.181:0/3111322643 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.853+0000 7f6227fff640 1 --1- 172.21.15.181:0/3111322643 >> v1:172.21.15.187:6789/0 conn(0x7f62280b7950 0x7f62280d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:40646/0 (socket says 172.21.15.181:40646) 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f62277fe640 1 -- 172.21.15.181:0/3111322643 >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 msgr2=0x7f622815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f62277fe640 1 --2- 172.21.15.181:0/3111322643 >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 0x7f622815fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f62277fe640 1 -- 172.21.15.181:0/3111322643 >> v1:172.21.15.187:6789/0 conn(0x7f62280b7950 legacy=0x7f62280d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f62277fe640 1 -- 172.21.15.181:0/3111322643 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622815cc00 con 0x7f62280d1980 2024-03-29T04:11:26.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f622cf1f640 1 --2- 172.21.15.181:0/3111322643 >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 0x7f622815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:26.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.854+0000 7f62277fe640 1 --2- 172.21.15.181:0/3111322643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f621c009870 tx=0x7f621c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=974fb74f07dad096 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f6226ffd640 1 -- 172.21.15.181:0/3111322643 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c034070 con 0x7f62280d1980 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f6226ffd640 1 -- 172.21.15.181:0/3111322643 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621c030840 con 0x7f62280d1980 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f6226ffd640 1 -- 172.21.15.181:0/3111322643 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c030c10 con 0x7f62280d1980 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 -- 172.21.15.181:0/3111322643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 msgr2=0x7f622815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 --2- 172.21.15.181:0/3111322643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f621c009870 tx=0x7f621c02b980 comp rx=0 tx=0).stop 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 -- 172.21.15.181:0/3111322643 shutdown_connections 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 --2- 172.21.15.181:0/3111322643 >> v2:172.21.15.187:3300/0 conn(0x7f6228071aa0 0x7f622815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 --2- 172.21.15.181:0/3111322643 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f622815d400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 -- 172.21.15.181:0/3111322643 >> 172.21.15.181:0/3111322643 conn(0x7f62280b3cb0 msgr2=0x7f62280b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 -- 172.21.15.181:0/3111322643 shutdown_connections 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.855+0000 7f622e982640 1 -- 172.21.15.181:0/3111322643 wait complete. 2024-03-29T04:11:26.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.856+0000 7f622e982640 1 Processor -- start 2024-03-29T04:11:26.837 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 -- start start 2024-03-29T04:11:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 0x7f62280b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f62280c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6228070790 con 0x7f6228071aa0 2024-03-29T04:11:26.839 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62280b7d30 con 0x7f62280b7950 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f622e982640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62280cc7b0 con 0x7f62280d1980 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f6227fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f6227fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56150/0 (socket says 172.21.15.181:56150) 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.857+0000 7f6227fff640 1 -- 172.21.15.181:0/1825149307 learned_addr learned my addr 172.21.15.181:0/1825149307 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f62277fe640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 0x7f62280b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f622cf1f640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f62280c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 -- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 msgr2=0x7f62280c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f62280c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 -- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 msgr2=0x7f62280b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 0x7f62280b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 -- 172.21.15.181:0/1825149307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62280ca340 con 0x7f6228071aa0 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f62277fe640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 0x7f62280b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f622cf1f640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f62280c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6227fff640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f621800c400 tx=0x7f621800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.840 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62180074e0 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6218007680 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62280ca620 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218004a00 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.858+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62280b2b20 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.859+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228075890 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.860+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f621801a020 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.860+0000 7f6224ff9640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 0x7f621406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.860+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f621808b060 con 0x7f6228071aa0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.860+0000 7f62277fe640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 0x7f621406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:26.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.861+0000 7f62277fe640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 0x7f621406ea20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f621c036000 tx=0x7f621c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:26.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:26.865+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6218010300 con 0x7f6228071aa0 2024-03-29T04:11:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.045+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6228070320 con 0x7f6228071aa0 2024-03-29T04:11:27.026 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.045+0000 7f6224ff9640 1 -- 172.21.15.181:0/1825149307 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f62180594d0 con 0x7f6228071aa0 2024-03-29T04:11:27.026 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 32 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 msgr2=0x7f621406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 0x7f621406ea20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f621c036000 tx=0x7f621c03b040 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 msgr2=0x7f62280c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f621800c400 tx=0x7f621800c8c0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 shutdown_connections 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f621406c560 0x7f621406ea20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62280d1980 0x7f62280c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62280b7950 0x7f62280b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 --2- 172.21.15.181:0/1825149307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228071aa0 0x7f62280c7a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 >> 172.21.15.181:0/1825149307 conn(0x7f62280b3cb0 msgr2=0x7f622815a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 shutdown_connections 2024-03-29T04:11:27.028 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.048+0000 7f622e982640 1 -- 172.21.15.181:0/1825149307 wait complete. 2024-03-29T04:11:27.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T04:11:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.164+0000 7f9b82416640 1 Processor -- start 2024-03-29T04:11:27.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.164+0000 7f9b82416640 1 -- start start 2024-03-29T04:11:27.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b82416640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 0x7f9b7c162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b82416640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b82416640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7c1630b0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b82416640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7c073880 con 0x7f9b7c15e5b0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b82416640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d0e20 con 0x7f9b7c073310 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b809b3640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9b7c0d1be0 0x7f9b7c0b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:40648/0 (socket says 172.21.15.181:40648) 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b809b3640 1 -- 172.21.15.181:0/1436358125 learned_addr learned my addr 172.21.15.181:0/1436358125 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1436358125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7bfff640 1 --2- 172.21.15.181:0/1436358125 >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 0x7f9b7c162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7affd640 1 -- 172.21.15.181:0/1436358125 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509298297 0 0) 0x7f9b7c1630b0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7affd640 1 -- 172.21.15.181:0/1436358125 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b64003520 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1436358125 >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 msgr2=0x7f9b7c162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1436358125 >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 0x7f9b7c162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1436358125 >> v1:172.21.15.187:6789/0 conn(0x7f9b7c0d1be0 legacy=0x7f9b7c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.145 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1436358125 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c0d1100 con 0x7f9b7c073310 2024-03-29T04:11:27.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.165+0000 7f9b7bfff640 1 --2- 172.21.15.181:0/1436358125 >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 0x7f9b7c162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.166+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1436358125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c06b580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0097a0 tx=0x7f9b6c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28929a2abf44fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.166+0000 7f9b7affd640 1 -- 172.21.15.181:0/1436358125 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0315a0 con 0x7f9b7c073310 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.166+0000 7f9b7affd640 1 -- 172.21.15.181:0/1436358125 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c037070 con 0x7f9b7c073310 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.166+0000 7f9b7affd640 1 -- 172.21.15.181:0/1436358125 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c031960 con 0x7f9b7c073310 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 -- 172.21.15.181:0/1436358125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 msgr2=0x7f9b7c06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 --2- 172.21.15.181:0/1436358125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c06b580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0097a0 tx=0x7f9b6c02b840 comp rx=0 tx=0).stop 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 -- 172.21.15.181:0/1436358125 shutdown_connections 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 --2- 172.21.15.181:0/1436358125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c06b580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 --2- 172.21.15.181:0/1436358125 >> v2:172.21.15.187:3300/0 conn(0x7f9b7c15e5b0 0x7f9b7c162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 -- 172.21.15.181:0/1436358125 >> 172.21.15.181:0/1436358125 conn(0x7f9b7c06e720 msgr2=0x7f9b7c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 -- 172.21.15.181:0/1436358125 shutdown_connections 2024-03-29T04:11:27.147 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.167+0000 7f9b82416640 1 -- 172.21.15.181:0/1436358125 wait complete. 2024-03-29T04:11:27.149 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.168+0000 7f9b82416640 1 Processor -- start 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.168+0000 7f9b82416640 1 -- start start 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 0x7f9b7c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c15e990 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d16c0 con 0x7f9b7c15e5b0 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b82416640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c06bac0 con 0x7f9b7c073310 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56166/0 (socket says 172.21.15.181:56166) 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1205302781 learned_addr learned my addr 172.21.15.181:0/1205302781 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b809b3640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 0x7f9b7c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7bfff640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 msgr2=0x7f9b7c0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.150 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c0b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 msgr2=0x7f9b7c148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 0x7f9b7c148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b7b7fe640 1 -- 172.21.15.181:0/1205302781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c0b35b0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.169+0000 7f9b809b3640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 0x7f9b7c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b7b7fe640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c034040 tx=0x7f9b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c032630 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0327d0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c032bc0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c0b3890 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.170+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c0b3e50 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.171+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c075890 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.171+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b6c04e430 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.172+0000 7f9b78ff9640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 0x7f9b6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.172+0000 7f9b7bfff640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 0x7f9b6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.172+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f9b6c02f030 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.172+0000 7f9b7bfff640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 0x7f9b6806e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009a30 tx=0x7f9b70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.155 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.176+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b6c040070 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.339 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.360+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b7c072be0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.340 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.360+0000 7f9b78ff9640 1 -- 172.21.15.181:0/1205302781 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f9b6c07bac0 con 0x7f9b7c0d1be0 2024-03-29T04:11:27.340 INFO:teuthology.orchestra.run.smithi181.stdout:pg_num: 64 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 msgr2=0x7f9b6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 0x7f9b6806e9f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009a30 tx=0x7f9b70008040 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 msgr2=0x7f9b7c146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c034040 tx=0x7f9b6c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 shutdown_connections 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b7c15e5b0 0x7f9b7c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b7c0d1be0 0x7f9b7c146250 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f9b6806c530 0x7f9b6806e9f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 --2- 172.21.15.181:0/1205302781 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b7c073310 0x7f9b7c0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 >> 172.21.15.181:0/1205302781 conn(0x7f9b7c06e720 msgr2=0x7f9b7c06bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 shutdown_connections 2024-03-29T04:11:27.343 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.362+0000 7f9b82416640 1 -- 172.21.15.181:0/1205302781 wait complete. 2024-03-29T04:11:27.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.475+0000 7f2b6de37640 1 Processor -- start 2024-03-29T04:11:27.454 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.475+0000 7f2b6de37640 1 -- start start 2024-03-29T04:11:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.476+0000 7f2b6de37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.455 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.476+0000 7f2b6de37640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b6806f130 0x7f2b680b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.476+0000 7f2b6de37640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6815c680 con 0x7f2b6815ed30 2024-03-29T04:11:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.476+0000 7f2b6de37640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6806bef0 con 0x7f2b6806f130 2024-03-29T04:11:27.456 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.476+0000 7f2b6de37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6806c1d0 con 0x7f2b6815c2a0 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b67fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b6806f130 0x7f2b680b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b677fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:55602/0 (socket says 172.21.15.181:55602) 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b677fe640 1 -- 172.21.15.181:0/1775472890 learned_addr learned my addr 172.21.15.181:0/1775472890 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b67fff640 1 -- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 msgr2=0x7f2b6806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b67fff640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b67fff640 1 -- 172.21.15.181:0/1775472890 >> v1:172.21.15.187:6789/0 conn(0x7f2b6815ed30 legacy=0x7f2b6806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b67fff640 1 -- 172.21.15.181:0/1775472890 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6806c4b0 con 0x7f2b6806f130 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b667fc640 1 -- 172.21.15.181:0/1775472890 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390737253 0 0) 0x7f2b6815c680 con 0x7f2b6815ed30 2024-03-29T04:11:27.457 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.477+0000 7f2b677fe640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.478+0000 7f2b67fff640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6806f130 0x7f2b680b4240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009d90 tx=0x7f2b6002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc252593e4fa66f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.478+0000 7f2b667fc640 1 -- 172.21.15.181:0/1775472890 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60039070 con 0x7f2b6806f130 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.478+0000 7f2b667fc640 1 -- 172.21.15.181:0/1775472890 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6002cb30 con 0x7f2b6806f130 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.478+0000 7f2b667fc640 1 -- 172.21.15.181:0/1775472890 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60034440 con 0x7f2b6806f130 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 -- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6806f130 msgr2=0x7f2b680b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6806f130 0x7f2b680b4240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009d90 tx=0x7f2b6002bd80 comp rx=0 tx=0).stop 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 -- 172.21.15.181:0/1775472890 shutdown_connections 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6806f130 0x7f2b680b4240 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1775472890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6815c2a0 0x7f2b6806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 -- 172.21.15.181:0/1775472890 >> 172.21.15.181:0/1775472890 conn(0x7f2b680d0af0 msgr2=0x7f2b680d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 -- 172.21.15.181:0/1775472890 shutdown_connections 2024-03-29T04:11:27.459 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.479+0000 7f2b6de37640 1 -- 172.21.15.181:0/1775472890 wait complete. 2024-03-29T04:11:27.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.481+0000 7f2b6de37640 1 Processor -- start 2024-03-29T04:11:27.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.481+0000 7f2b6de37640 1 -- start start 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6806f130 0x7f2b6815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 0x7f2b680cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6806bc40 con 0x7f2b6815c2a0 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6806bac0 con 0x7f2b6815ed30 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b6de37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b680b4780 con 0x7f2b6806f130 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.482+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:56170/0 (socket says 172.21.15.181:56170) 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 -- 172.21.15.181:0/1686501037 learned_addr learned my addr 172.21.15.181:0/1686501037 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b677fe640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6806f130 0x7f2b6815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b67fff640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 0x7f2b680cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 -- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6806f130 msgr2=0x7f2b6815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6806f130 0x7f2b6815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 -- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 msgr2=0x7f2b680cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 0x7f2b680cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 -- 172.21.15.181:0/1686501037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680cdab0 con 0x7f2b6815c2a0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b67fff640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 0x7f2b680cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b66ffd640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c440 tx=0x7f2b5c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c017070 con 0x7f2b6815c2a0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c005480 con 0x7f2b6815c2a0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c003830 con 0x7f2b6815c2a0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b680cdd90 con 0x7f2b6815c2a0 2024-03-29T04:11:27.464 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.483+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b680ce350 con 0x7f2b6815c2a0 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.484+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b68075a10 con 0x7f2b6815c2a0 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.485+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2b5c01a070 con 0x7f2b6815c2a0 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.485+0000 7f2b4ffff640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 0x7f2b5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.485+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f2b5c08aa10 con 0x7f2b6815c2a0 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.485+0000 7f2b677fe640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 0x7f2b5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.486+0000 7f2b677fe640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 0x7f2b5406e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009630 tx=0x7f2b58009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.489+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2b5c0588c0 con 0x7f2b6815c2a0 2024-03-29T04:11:27.687 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.708+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b680b4020 con 0x7f2b6815c2a0 2024-03-29T04:11:27.688 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.709+0000 7f2b4ffff640 1 -- 172.21.15.181:0/1686501037 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f2b5c08a070 con 0x7f2b6815c2a0 2024-03-29T04:11:27.688 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:27.688 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:44.376045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:11:27.689 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 18 2024-03-29T04:11:27.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 msgr2=0x7f2b5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 0x7f2b5406e9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009630 tx=0x7f2b58009240 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 msgr2=0x7f2b680ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00c440 tx=0x7f2b5c00c900 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 shutdown_connections 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b6815ed30 0x7f2b680cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b6815c2a0 0x7f2b680ceb50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2b5406c530 0x7f2b5406e9f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 --2- 172.21.15.181:0/1686501037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b6806f130 0x7f2b6815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 >> 172.21.15.181:0/1686501037 conn(0x7f2b680d0af0 msgr2=0x7f2b680b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 shutdown_connections 2024-03-29T04:11:27.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.711+0000 7f2b6de37640 1 -- 172.21.15.181:0/1686501037 wait complete. 2024-03-29T04:11:27.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 Processor -- start 2024-03-29T04:11:27.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 -- start start 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5b015e950 0x7fa5b00d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d30d0 0x7fa5b00b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b00d36f0 con 0x7fa5b00d48c0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b00d3cb0 con 0x7fa5b015e950 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.824+0000 7fa5b6996640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00d4d00 con 0x7fa5b00d30d0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5affff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5b015e950 0x7fa5b00d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5af7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa5b00d48c0 0x7fa5b00d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38106/0 (socket says 172.21.15.181:38106) 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5af7fe640 1 -- 172.21.15.181:0/1360460496 learned_addr learned my addr 172.21.15.181:0/1360460496 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:27.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5b4f33640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d30d0 0x7fa5b00b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5affff640 1 -- 172.21.15.181:0/1360460496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d30d0 msgr2=0x7fa5b00b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5affff640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d30d0 0x7fa5b00b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5affff640 1 -- 172.21.15.181:0/1360460496 >> v1:172.21.15.181:6789/0 conn(0x7fa5b00d48c0 legacy=0x7fa5b00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5affff640 1 -- 172.21.15.181:0/1360460496 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b00d39d0 con 0x7fa5b015e950 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.825+0000 7fa5aeffd640 1 -- 172.21.15.181:0/1360460496 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318054644 0 0) 0x7fa5b00d36f0 con 0x7fa5b00d48c0 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5affff640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b015e950 0x7fa5b00d4380 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240bda2cfc12c048 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5aeffd640 1 -- 172.21.15.181:0/1360460496 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0034070 con 0x7fa5b015e950 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5aeffd640 1 -- 172.21.15.181:0/1360460496 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0030610 con 0x7fa5b015e950 2024-03-29T04:11:27.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5aeffd640 1 -- 172.21.15.181:0/1360460496 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00309e0 con 0x7fa5b015e950 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 -- 172.21.15.181:0/1360460496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b015e950 msgr2=0x7fa5b00d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b015e950 0x7fa5b00d4380 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 -- 172.21.15.181:0/1360460496 shutdown_connections 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d30d0 0x7fa5b00b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1360460496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b015e950 0x7fa5b00d4380 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 -- 172.21.15.181:0/1360460496 >> 172.21.15.181:0/1360460496 conn(0x7fa5b00676e0 msgr2=0x7fa5b0067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 -- 172.21.15.181:0/1360460496 shutdown_connections 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.826+0000 7fa5b6996640 1 -- 172.21.15.181:0/1360460496 wait complete. 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.827+0000 7fa5b6996640 1 Processor -- start 2024-03-29T04:11:27.808 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.828+0000 7fa5b6996640 1 -- start start 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d48c0 0x7fa5b015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b015e950 0x7fa5b014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b015ed30 con 0x7fa5b00d30d0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00b7da0 con 0x7fa5b015e950 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b6996640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b015eeb0 con 0x7fa5b00d48c0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60500/0 (socket says 172.21.15.181:60500) 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 -- 172.21.15.181:0/1793621737 learned_addr learned my addr 172.21.15.181:0/1793621737 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5af7fe640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d48c0 0x7fa5b015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.811 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5b4f33640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b015e950 0x7fa5b014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 -- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d48c0 msgr2=0x7fa5b015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d48c0 0x7fa5b015abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 -- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b015e950 msgr2=0x7fa5b014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b015e950 0x7fa5b014e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.829+0000 7fa5affff640 1 -- 172.21.15.181:0/1793621737 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b015b230 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5affff640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000c00 tx=0x7fa5a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0047070 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0030410 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b00c94a0 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0034040 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.830+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b00c9a60 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.831+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b00c9290 con 0x7fa5b00d30d0 2024-03-29T04:11:27.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.831+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa5a0037070 con 0x7fa5b00d30d0 2024-03-29T04:11:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.831+0000 7fa5acff9640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 0x7fa59c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.831+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fa5a00ad020 con 0x7fa5b00d30d0 2024-03-29T04:11:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.832+0000 7fa5af7fe640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 0x7fa59c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:27.813 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.832+0000 7fa5af7fe640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 0x7fa59c06ea20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5b01f8040 tx=0x7fa5a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:27.815 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:27.836+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa5a00305e0 con 0x7fa5b00d30d0 2024-03-29T04:11:27.997 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.018+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5b0075a10 con 0x7fa5b00d30d0 2024-03-29T04:11:27.998 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.019+0000 7fa5acff9640 1 -- 172.21.15.181:0/1793621737 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7fa5a007b410 con 0x7fa5b00d30d0 2024-03-29T04:11:27.998 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:27.998 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":41,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:58.390325+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 msgr2=0x7fa59c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.000 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 0x7fa59c06ea20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5b01f8040 tx=0x7fa5a4009040 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 msgr2=0x7fa5b0158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000c00 tx=0x7fa5a00046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 shutdown_connections 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b015e950 0x7fa5b014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa59c06c560 0x7fa59c06ea20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b00d48c0 0x7fa5b015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 --2- 172.21.15.181:0/1793621737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00d30d0 0x7fa5b0158090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 >> 172.21.15.181:0/1793621737 conn(0x7fa5b00676e0 msgr2=0x7fa5b00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 shutdown_connections 2024-03-29T04:11:28.001 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.021+0000 7fa5b6996640 1 -- 172.21.15.181:0/1793621737 wait complete. 2024-03-29T04:11:28.012 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:11:28.013 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 Processor -- start 2024-03-29T04:11:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 -- start start 2024-03-29T04:11:28.114 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa83815e950 0x7fa8380d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8380d48c0 0x7fa8380d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8380d36f0 con 0x7fa8380d2f00 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fa8380d3cb0 con 0x7fa8380d48c0 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.134+0000 7fa83cc3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8380d4d00 con 0x7fa83815e950 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa82ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8380d48c0 0x7fa8380d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa836d9d640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa8380d2f00 0x7fa8380cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38110/0 (socket says 172.21.15.181:38110) 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa836d9d640 1 -- 172.21.15.181:0/1091753570 learned_addr learned my addr 172.21.15.181:0/1091753570 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa83659c640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa83815e950 0x7fa8380d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa82ffff640 1 -- 172.21.15.181:0/1091753570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa83815e950 msgr2=0x7fa8380d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa82ffff640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa83815e950 0x7fa8380d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa82ffff640 1 -- 172.21.15.181:0/1091753570 >> v1:172.21.15.181:6789/0 conn(0x7fa8380d2f00 legacy=0x7fa8380cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa835d9b640 1 -- 172.21.15.181:0/1091753570 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015413722 0 0) 0x7fa8380d36f0 con 0x7fa8380d2f00 2024-03-29T04:11:28.115 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.135+0000 7fa82ffff640 1 -- 172.21.15.181:0/1091753570 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8380d39d0 con 0x7fa8380d48c0 2024-03-29T04:11:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa82ffff640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8380d48c0 0x7fa8380d29c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa8200097a0 tx=0x7fa82002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d602972f302967f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa835d9b640 1 -- 172.21.15.181:0/1091753570 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820034070 con 0x7fa8380d48c0 2024-03-29T04:11:28.117 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa835d9b640 1 -- 172.21.15.181:0/1091753570 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820030610 con 0x7fa8380d48c0 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa835d9b640 1 -- 172.21.15.181:0/1091753570 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200309e0 con 0x7fa8380d48c0 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 -- 172.21.15.181:0/1091753570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8380d48c0 msgr2=0x7fa8380d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8380d48c0 0x7fa8380d29c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa8200097a0 tx=0x7fa82002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 -- 172.21.15.181:0/1091753570 shutdown_connections 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8380d48c0 0x7fa8380d29c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/1091753570 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa83815e950 0x7fa8380d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 -- 172.21.15.181:0/1091753570 >> 172.21.15.181:0/1091753570 conn(0x7fa8380676e0 msgr2=0x7fa838067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 -- 172.21.15.181:0/1091753570 shutdown_connections 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.136+0000 7fa83cc3c640 1 -- 172.21.15.181:0/1091753570 wait complete. 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 Processor -- start 2024-03-29T04:11:28.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 -- start start 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8380d2f00 0x7fa838149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8380d48c0 0x7fa838160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8380d3390 con 0x7fa83815e950 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8380b7da0 con 0x7fa8380d2f00 2024-03-29T04:11:28.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.138+0000 7fa83cc3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815ed30 con 0x7fa8380d48c0 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60516/0 (socket says 172.21.15.181:60516) 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 -- 172.21.15.181:0/3090128588 learned_addr learned my addr 172.21.15.181:0/3090128588 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 -- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8380d48c0 msgr2=0x7fa838160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa82ffff640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8380d48c0 0x7fa838160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa83659c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8380d2f00 0x7fa838149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8380d48c0 0x7fa838160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 -- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8380d2f00 msgr2=0x7fa838149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8380d2f00 0x7fa838149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 -- 172.21.15.181:0/3090128588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8380cabd0 con 0x7fa83815e950 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa836d9d640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c700 tx=0x7fa83000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830018070 con 0x7fa83815e950 2024-03-29T04:11:28.126 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83000ce10 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8380c8ad0 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83000a860 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.139+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8380c9090 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.140+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838075a10 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.141+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa83001b070 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.141+0000 7fa82effd640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.141+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fa83008ad60 con 0x7fa83815e950 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.141+0000 7fa83659c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.142+0000 7fa83659c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 0x7fa81806e9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa828006fd0 tx=0x7fa828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.146+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa830059260 con 0x7fa83815e950 2024-03-29T04:11:28.337 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.358+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8380b9770 con 0x7fa83815e950 2024-03-29T04:11:28.338 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.359+0000 7fa82effd640 1 -- 172.21.15.181:0/3090128588 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7fa830058b60 con 0x7fa83815e950 2024-03-29T04:11:28.338 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:28.338 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:44.376045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:11:28.339 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 18 2024-03-29T04:11:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 msgr2=0x7fa81806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 0x7fa81806e9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa828006fd0 tx=0x7fa828008040 comp rx=0 tx=0).stop 2024-03-29T04:11:28.341 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 msgr2=0x7fa838161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c700 tx=0x7fa83000cbc0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 shutdown_connections 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83815e950 0x7fa838161520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8380d48c0 0x7fa838160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 --2- 172.21.15.181:0/3090128588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8380d2f00 0x7fa838149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 >> 172.21.15.181:0/3090128588 conn(0x7fa8380676e0 msgr2=0x7fa8380cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 shutdown_connections 2024-03-29T04:11:28.342 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.362+0000 7fa83cc3c640 1 -- 172.21.15.181:0/3090128588 wait complete. 2024-03-29T04:11:28.356 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.479+0000 7f5394288640 1 Processor -- start 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.479+0000 7f5394288640 1 -- start start 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5394288640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5394288640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 0x7f538c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5394288640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538c15b650 con 0x7f538c0ccf10 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5394288640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f538c072ab0 con 0x7f538c072280 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5394288640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f538c072d90 con 0x7f538c15b270 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5391823640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 0x7f538c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:44440/0 (socket says 172.21.15.181:44440) 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 -- 172.21.15.181:0/3422152623 learned_addr learned my addr 172.21.15.181:0/3422152623 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:28.460 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 -- 172.21.15.181:0/3422152623 >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 msgr2=0x7f538c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 --2- 172.21.15.181:0/3422152623 >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 0x7f538c0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 -- 172.21.15.181:0/3422152623 >> v1:172.21.15.187:6789/0 conn(0x7f538c0ccf10 legacy=0x7f538c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5392024640 1 -- 172.21.15.181:0/3422152623 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c073070 con 0x7f538c15b270 2024-03-29T04:11:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5391022640 1 -- 172.21.15.181:0/3422152623 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837325759 0 0) 0x7f538c15b650 con 0x7f538c0ccf10 2024-03-29T04:11:28.461 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.480+0000 7f5391823640 1 --2- 172.21.15.181:0/3422152623 >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 0x7f538c0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.481+0000 7f5392024640 1 --2- 172.21.15.181:0/3422152623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5388009870 tx=0x7f538802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c295c0c660f26351 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.481+0000 7f5391022640 1 -- 172.21.15.181:0/3422152623 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388034070 con 0x7f538c15b270 2024-03-29T04:11:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.481+0000 7f5391022640 1 -- 172.21.15.181:0/3422152623 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5388030840 con 0x7f538c15b270 2024-03-29T04:11:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.481+0000 7f5391022640 1 -- 172.21.15.181:0/3422152623 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388030c10 con 0x7f538c15b270 2024-03-29T04:11:28.462 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 -- 172.21.15.181:0/3422152623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 msgr2=0x7f538c15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 --2- 172.21.15.181:0/3422152623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5388009870 tx=0x7f538802b980 comp rx=0 tx=0).stop 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 -- 172.21.15.181:0/3422152623 shutdown_connections 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 --2- 172.21.15.181:0/3422152623 >> v2:172.21.15.187:3300/0 conn(0x7f538c072280 0x7f538c0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 --2- 172.21.15.181:0/3422152623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c15b270 0x7f538c15f7e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 -- 172.21.15.181:0/3422152623 >> 172.21.15.181:0/3422152623 conn(0x7f538c0b1c90 msgr2=0x7f538c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 -- 172.21.15.181:0/3422152623 shutdown_connections 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.482+0000 7f5394288640 1 -- 172.21.15.181:0/3422152623 wait complete. 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.483+0000 7f5394288640 1 Processor -- start 2024-03-29T04:11:28.463 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.483+0000 7f5394288640 1 -- start start 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f538c072280 0x7f538c0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c0ccf10 0x7f538c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f538c0726e0 con 0x7f538c15b270 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f538c073630 con 0x7f538c072280 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5394288640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f538c072860 con 0x7f538c0ccf10 2024-03-29T04:11:28.465 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60518/0 (socket says 172.21.15.181:60518) 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 -- 172.21.15.181:0/2604485180 learned_addr learned my addr 172.21.15.181:0/2604485180 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5391823640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c0ccf10 0x7f538c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392024640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f538c072280 0x7f538c0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 -- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c0ccf10 msgr2=0x7f538c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c0ccf10 0x7f538c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 -- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f538c072280 msgr2=0x7f538c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f538c072280 0x7f538c0c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.484+0000 7f5392825640 1 -- 172.21.15.181:0/2604485180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c159540 con 0x7f538c15b270 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f5392825640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f538400c760 tx=0x7f538400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384003da0 con 0x7f538c15b270 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384005070 con 0x7f538c15b270 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c159820 con 0x7f538c15b270 2024-03-29T04:11:28.466 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384005460 con 0x7f538c15b270 2024-03-29T04:11:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.485+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c159de0 con 0x7f538c15b270 2024-03-29T04:11:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.486+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c1592b0 con 0x7f538c15b270 2024-03-29T04:11:28.467 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.486+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5384005600 con 0x7f538c15b270 2024-03-29T04:11:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.486+0000 7f537effd640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 0x7f537006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.487+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f538408b5f0 con 0x7f538c15b270 2024-03-29T04:11:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.487+0000 7f5392024640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 0x7f537006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.468 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.487+0000 7f5392024640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 0x7f537006e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5388036000 tx=0x7f538803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.470 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.491+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f53840594a0 con 0x7f538c15b270 2024-03-29T04:11:28.690 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.711+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f538c075a10 con 0x7f538c15b270 2024-03-29T04:11:28.691 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.712+0000 7f537effd640 1 -- 172.21.15.181:0/2604485180 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5193 (secure 0 0 0) 0x7f538405d7a0 con 0x7f538c15b270 2024-03-29T04:11:28.691 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:28.691 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T04:10:22.645810+0000","modified":"2024-03-29T04:10:44.376045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5373":{"gid":5373,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2447436157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2447436157},{"type":"v1","addr":"172.21.15.181:6837","nonce":2447436157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5373,"qdb_cluster":[5373]},"id":2}]} 2024-03-29T04:11:28.692 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 18 2024-03-29T04:11:28.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 msgr2=0x7f537006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.693 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 0x7f537006e9f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5388036000 tx=0x7f538803c040 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 msgr2=0x7f538c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f538400c760 tx=0x7f538400cc20 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 shutdown_connections 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f538c15b270 0x7f538c0b4e70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f537006c530 0x7f537006e9f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f538c0ccf10 0x7f538c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 --2- 172.21.15.181:0/2604485180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f538c072280 0x7f538c0c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 >> 172.21.15.181:0/2604485180 conn(0x7f538c0b1c90 msgr2=0x7f538c15c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 shutdown_connections 2024-03-29T04:11:28.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.714+0000 7f5394288640 1 -- 172.21.15.181:0/2604485180 wait complete. 2024-03-29T04:11:28.704 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.843+0000 7f40725d4640 1 Processor -- start 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.843+0000 7f40725d4640 1 -- start start 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f40725d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f406c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f40725d4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c15b650 con 0x7f406c15b270 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f40725d4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c072ab0 con 0x7f406c0ccf10 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f40725d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f406c15fc40 con 0x7f406c15f7e0 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406bfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f406c15b270 0x7f406c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38114/0 (socket says 172.21.15.181:38114) 2024-03-29T04:11:28.823 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406bfff640 1 -- 172.21.15.181:0/798358517 learned_addr learned my addr 172.21.15.181:0/798358517 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278110691 0 0) 0x7f406c15b650 con 0x7f406c15b270 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406b7fe640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f406c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4054003520 con 0x7f406c15b270 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1231627707 0 0) 0x7f4054003520 con 0x7f406c15b270 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 msgr2=0x7f406c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.826 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f406c0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 >> v1:172.21.15.187:6789/0 conn(0x7f406c0ccf10 legacy=0x7f406c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c072d90 con 0x7f406c15b270 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f4060001670 con 0x7f406c15b270 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.844+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 >> v1:172.21.15.181:6789/0 conn(0x7f406c15b270 legacy=0x7f406c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40540039e0 0x7f4054003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f4054004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f40540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f406c15fc40 con 0x7f4054004d40 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f405400dc80 con 0x7f40540039e0 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f405400de80 con 0x7f406c15f7e0 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f40540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406bfff640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40540039e0 0x7f4054003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f406b7fe640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f4054004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 -- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 msgr2=0x7f4054004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c15f7e0 0x7f4054004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 -- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40540039e0 msgr2=0x7f4054003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40540039e0 0x7f4054003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.845+0000 7f4070b71640 1 -- 172.21.15.181:0/798358517 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4054003520 con 0x7f4054004d40 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f4070b71640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f40540051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4064001e20 tx=0x7f4054005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c2c9f80da996d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064001a70 con 0x7f4054004d40 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4064003b70 con 0x7f4054004d40 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f406affd640 1 -- 172.21.15.181:0/798358517 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640050d0 con 0x7f4054004d40 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 -- 172.21.15.181:0/798358517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 msgr2=0x7f40540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 --2- 172.21.15.181:0/798358517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f40540051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4064001e20 tx=0x7f4054005830 comp rx=0 tx=0).stop 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f4070b71640 1 -- 172.21.15.181:0/798358517 reap_dead start 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 -- 172.21.15.181:0/798358517 shutdown_connections 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 -- 172.21.15.181:0/798358517 >> 172.21.15.181:0/798358517 conn(0x7f406c0b1c90 msgr2=0x7f406c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 -- 172.21.15.181:0/798358517 shutdown_connections 2024-03-29T04:11:28.828 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.846+0000 7f40725d4640 1 -- 172.21.15.181:0/798358517 wait complete. 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 Processor -- start 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 -- start start 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c1561d0 0x7f406c0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c156650 0x7f406c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.848+0000 7f40725d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f406c073630 con 0x7f4054004d40 2024-03-29T04:11:28.831 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f40725d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f406c073350 con 0x7f406c1561d0 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f40725d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0cc9d0 con 0x7f406c156650 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60544/0 (socket says 172.21.15.181:60544) 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 -- 172.21.15.181:0/4051140722 learned_addr learned my addr 172.21.15.181:0/4051140722 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 -- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c156650 msgr2=0x7f406c0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c156650 0x7f406c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 -- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c1561d0 msgr2=0x7f406c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c1561d0 0x7f406c0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 -- 172.21.15.181:0/4051140722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0c9cb0 con 0x7f4054004d40 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f406bfff640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4060007290 tx=0x7f4060007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000f9c0 con 0x7f4054004d40 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4060018070 con 0x7f4054004d40 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000fda0 con 0x7f4054004d40 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c0b2bd0 con 0x7f4054004d40 2024-03-29T04:11:28.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.849+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c0b4be0 con 0x7f4054004d40 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.851+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c075a10 con 0x7f4054004d40 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.851+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f406001d460 con 0x7f4054004d40 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.851+0000 7f4068ff9640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 0x7f405806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.851+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f406008b900 con 0x7f4054004d40 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.851+0000 7f406b7fe640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 0x7f405806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:28.833 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.852+0000 7f406b7fe640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 0x7f405806e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f406c0ca2e0 tx=0x7f405c0095d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:28.834 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:28.856+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4060059d70 con 0x7f4054004d40 2024-03-29T04:11:29.017 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.038+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f406c0ce8a0 con 0x7f4054004d40 2024-03-29T04:11:29.018 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.039+0000 7f4068ff9640 1 -- 172.21.15.181:0/4051140722 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17003 (secure 0 0 0) 0x7f4060059670 con 0x7f4054004d40 2024-03-29T04:11:29.018 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:29.018 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":41,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:10:58.390325+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T04:10:20.863137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T04:10:21.635882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:29.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 msgr2=0x7f405806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.020 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 0x7f405806e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f406c0ca2e0 tx=0x7f405c0095d0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 msgr2=0x7f406c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4060007290 tx=0x7f4060007750 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 shutdown_connections 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f405806c530 0x7f405806e9f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f406c156650 0x7f406c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f406c1561d0 0x7f406c0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 --2- 172.21.15.181:0/4051140722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d40 0x7f406c0b3690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 >> 172.21.15.181:0/4051140722 conn(0x7f406c0b1c90 msgr2=0x7f406c157150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 shutdown_connections 2024-03-29T04:11:29.021 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.041+0000 7f40725d4640 1 -- 172.21.15.181:0/4051140722 wait complete. 2024-03-29T04:11:29.032 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T04:11:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.155+0000 7f5653b9d640 1 Processor -- start 2024-03-29T04:11:29.134 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.155+0000 7f5653b9d640 1 -- start start 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5653b9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f564c15e7a0 0x7f564c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5653b9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cec60 0x7f564c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5653b9d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564c0d3120 con 0x7f564c0d3ee0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5653b9d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f564c0cfc40 con 0x7f564c15e7a0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5653b9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f564c0d3400 con 0x7f564c0cec60 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f565213a640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f564c0d3ee0 0x7f564c0b5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38120/0 (socket says 172.21.15.181:38120) 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f565213a640 1 -- 172.21.15.181:0/2530481041 learned_addr learned my addr 172.21.15.181:0/2530481041 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651939640 1 --2- 172.21.15.181:0/2530481041 >> v2:172.21.15.187:3300/0 conn(0x7f564c15e7a0 0x7f564c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651138640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cec60 0x7f564c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5650937640 1 -- 172.21.15.181:0/2530481041 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 848682339 0 0) 0x7f564c0d3120 con 0x7f564c0d3ee0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5650937640 1 -- 172.21.15.181:0/2530481041 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5638003520 con 0x7f564c0d3ee0 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651939640 1 -- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cec60 msgr2=0x7f564c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651939640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cec60 0x7f564c0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651939640 1 -- 172.21.15.181:0/2530481041 >> v1:172.21.15.181:6789/0 conn(0x7f564c0d3ee0 legacy=0x7f564c0b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.137 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.156+0000 7f5651939640 1 -- 172.21.15.181:0/2530481041 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c0d36e0 con 0x7f564c15e7a0 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.157+0000 7f5651939640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c15e7a0 0x7f564c0cf3f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56480097a0 tx=0x7f564802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981a0096c5d3d836 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.157+0000 7f5650937640 1 -- 172.21.15.181:0/2530481041 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56480315a0 con 0x7f564c15e7a0 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.157+0000 7f5650937640 1 -- 172.21.15.181:0/2530481041 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648037070 con 0x7f564c15e7a0 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.157+0000 7f5650937640 1 -- 172.21.15.181:0/2530481041 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648031940 con 0x7f564c15e7a0 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 -- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c15e7a0 msgr2=0x7f564c0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c15e7a0 0x7f564c0cf3f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56480097a0 tx=0x7f564802b840 comp rx=0 tx=0).stop 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 -- 172.21.15.181:0/2530481041 shutdown_connections 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cec60 0x7f564c0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2530481041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c15e7a0 0x7f564c0cf3f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 -- 172.21.15.181:0/2530481041 >> 172.21.15.181:0/2530481041 conn(0x7f564c0676b0 msgr2=0x7f564c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 -- 172.21.15.181:0/2530481041 shutdown_connections 2024-03-29T04:11:29.138 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.158+0000 7f5653b9d640 1 -- 172.21.15.181:0/2530481041 wait complete. 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.159+0000 7f5653b9d640 1 Processor -- start 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.159+0000 7f5653b9d640 1 -- start start 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c0cec60 0x7f564c1f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cd600 0x7f564c14f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f564c0d17f0 con 0x7f564c0d3ee0 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f564c0cf930 con 0x7f564c0cec60 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5653b9d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f564c0d1970 con 0x7f564c0cd600 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60556/0 (socket says 172.21.15.181:60556) 2024-03-29T04:11:29.141 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 -- 172.21.15.181:0/2632738124 learned_addr learned my addr 172.21.15.181:0/2632738124 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651939640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c0cec60 0x7f564c1f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f565213a640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cd600 0x7f564c14f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 -- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cd600 msgr2=0x7f564c14f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cd600 0x7f564c14f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 -- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c0cec60 msgr2=0x7f564c1f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c0cec60 0x7f564c1f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 -- 172.21.15.181:0/2632738124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c1f7930 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.160+0000 7f5651138640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f563c00e9e0 tx=0x7f563c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c017070 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563c00c970 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c1f7c10 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c00cd60 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c14f9f0 con 0x7f564c0d3ee0 2024-03-29T04:11:29.142 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.161+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564c0cda10 con 0x7f564c0d3ee0 2024-03-29T04:11:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.162+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f563c01a070 con 0x7f564c0d3ee0 2024-03-29T04:11:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.163+0000 7f56427fc640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 0x7f563006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.163+0000 7f5651939640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 0x7f563006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.143 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.163+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f563c08ba60 con 0x7f564c0d3ee0 2024-03-29T04:11:29.144 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.163+0000 7f5651939640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 0x7f563006e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5648036000 tx=0x7f5648008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.167+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f564c0cda10 con 0x7f564c0d3ee0 2024-03-29T04:11:29.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.386+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f564c06d5f0 con 0x7f564c0d3ee0 2024-03-29T04:11:29.365 INFO:tasks.ceph.mon.a.smithi181.stderr:2024-03-29T04:11:29.386+0000 7f8224f14640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T04:11:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.388+0000 7f56427fc640 1 -- 172.21.15.181:0/2632738124 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f563c059910 con 0x7f564c0d3ee0 2024-03-29T04:11:29.367 INFO:teuthology.orchestra.run.smithi181.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T04:11:29.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 msgr2=0x7f563006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 0x7f563006e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5648036000 tx=0x7f5648008030 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 msgr2=0x7f564c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f563c00e9e0 tx=0x7f563c00eea0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 shutdown_connections 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f563006c530 0x7f563006e9f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f564c0cd600 0x7f564c14f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f564c0d3ee0 0x7f564c0cd0c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 --2- 172.21.15.181:0/2632738124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f564c0cec60 0x7f564c1f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 >> 172.21.15.181:0/2632738124 conn(0x7f564c0676b0 msgr2=0x7f564c15d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 shutdown_connections 2024-03-29T04:11:29.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.391+0000 7f5653b9d640 1 -- 172.21.15.181:0/2632738124 wait complete. 2024-03-29T04:11:29.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T04:11:29.408 INFO:tasks.ceph.mds.a.smithi181.stdout:starting mds.a at 2024-03-29T04:11:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.510+0000 7f18647ba640 1 Processor -- start 2024-03-29T04:11:29.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.510+0000 7f18647ba640 1 -- start start 2024-03-29T04:11:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f18647ba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f185c144cb0 0x7f185c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f18647ba640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f185c15f800 0x7f185c072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f18647ba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185c0b35e0 con 0x7f185c072b80 2024-03-29T04:11:29.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f18647ba640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f185c072fd0 con 0x7f185c15f800 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f18647ba640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f185c0b3e80 con 0x7f185c144cb0 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f185c15f800 0x7f185c072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f185c15f800 0x7f185c072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60590/0 (socket says 172.21.15.181:60590) 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 -- 172.21.15.181:0/606332469 learned_addr learned my addr 172.21.15.181:0/606332469 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1862556640 1 --2- 172.21.15.181:0/606332469 >> v2:172.21.15.187:3300/0 conn(0x7f185c144cb0 0x7f185c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 -- 172.21.15.181:0/606332469 >> v2:172.21.15.187:3300/0 conn(0x7f185c144cb0 msgr2=0x7f185c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 --2- 172.21.15.181:0/606332469 >> v2:172.21.15.187:3300/0 conn(0x7f185c144cb0 0x7f185c0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 -- 172.21.15.181:0/606332469 >> v1:172.21.15.187:6789/0 conn(0x7f185c072b80 legacy=0x7f185c071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.511+0000 7f1861d55640 1 -- 172.21.15.181:0/606332469 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c0b38c0 con 0x7f185c15f800 2024-03-29T04:11:29.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.512+0000 7f1861d55640 1 --2- 172.21.15.181:0/606332469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c15f800 0x7f185c072640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f185c06e3e0 tx=0x7f184c02dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da461dfb799fcd95 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.512+0000 7f1861554640 1 -- 172.21.15.181:0/606332469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c03b070 con 0x7f185c15f800 2024-03-29T04:11:29.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.512+0000 7f1861554640 1 -- 172.21.15.181:0/606332469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c02e960 con 0x7f185c15f800 2024-03-29T04:11:29.492 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.512+0000 7f1861554640 1 -- 172.21.15.181:0/606332469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c02ed30 con 0x7f185c15f800 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 -- 172.21.15.181:0/606332469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c15f800 msgr2=0x7f185c072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 --2- 172.21.15.181:0/606332469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c15f800 0x7f185c072640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f185c06e3e0 tx=0x7f184c02dbb0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 -- 172.21.15.181:0/606332469 shutdown_connections 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 --2- 172.21.15.181:0/606332469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c15f800 0x7f185c072640 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 --2- 172.21.15.181:0/606332469 >> v2:172.21.15.187:3300/0 conn(0x7f185c144cb0 0x7f185c0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 -- 172.21.15.181:0/606332469 >> 172.21.15.181:0/606332469 conn(0x7f185c0b5b00 msgr2=0x7f185c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 -- 172.21.15.181:0/606332469 shutdown_connections 2024-03-29T04:11:29.493 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.513+0000 7f18647ba640 1 -- 172.21.15.181:0/606332469 wait complete. 2024-03-29T04:11:29.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.514+0000 7f18647ba640 1 Processor -- start 2024-03-29T04:11:29.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.514+0000 7f18647ba640 1 -- start start 2024-03-29T04:11:29.495 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 0x7f185c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 0x7f185c14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f185c15fc40 con 0x7f185c144cb0 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f185c0cdb30 con 0x7f185c15f800 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f185c0cf060 con 0x7f185c072b80 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60604/0 (socket says 172.21.15.181:60604) 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 -- 172.21.15.181:0/2233386806 learned_addr learned my addr 172.21.15.181:0/2233386806 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1862556640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 0x7f185c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1862d57640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 0x7f185c14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 -- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 msgr2=0x7f185c14c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 0x7f185c14c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 -- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 msgr2=0x7f185c14b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:29.496 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 0x7f185c14b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 -- 172.21.15.181:0/2233386806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c149db0 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1862556640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 0x7f185c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1862d57640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 0x7f185c14b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1861d55640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f184c02dbf0 tx=0x7f184c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c03b040 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f184c046070 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c14a030 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c036d90 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.515+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c14a5f0 con 0x7f185c144cb0 2024-03-29T04:11:29.497 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.516+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c075890 con 0x7f185c144cb0 2024-03-29T04:11:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.517+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f184c04b460 con 0x7f185c144cb0 2024-03-29T04:11:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.517+0000 7f1852ffd640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 0x7f184006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.517+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7f184c0af890 con 0x7f185c144cb0 2024-03-29T04:11:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.517+0000 7f1862556640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 0x7f184006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:29.498 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.518+0000 7f1862556640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 0x7f184006e9f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1858009b90 tx=0x7f1858008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:29.500 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.522+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f185c149db0 con 0x7f185c144cb0 2024-03-29T04:11:29.713 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:29.734+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f185c0b4030 con 0x7f185c144cb0 2024-03-29T04:11:30.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.389+0000 7f1852ffd640 1 -- 172.21.15.181:0/2233386806 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f184c07d660 con 0x7f185c144cb0 2024-03-29T04:11:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 msgr2=0x7f184006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 0x7f184006e9f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1858009b90 tx=0x7f1858008040 comp rx=0 tx=0).stop 2024-03-29T04:11:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 msgr2=0x7f185c149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f184c02dbf0 tx=0x7f184c0046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.371 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 shutdown_connections 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f185c15f800 0x7f185c14b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f185c144cb0 0x7f185c149870 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f184006c530 0x7f184006e9f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 --2- 172.21.15.181:0/2233386806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f185c072b80 0x7f185c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 >> 172.21.15.181:0/2233386806 conn(0x7f185c0b5b00 msgr2=0x7f185c06d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 shutdown_connections 2024-03-29T04:11:30.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.392+0000 7f18647ba640 1 -- 172.21.15.181:0/2233386806 wait complete. 2024-03-29T04:11:30.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T04:11:30.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.556+0000 7fe9dc6e4640 1 Processor -- start 2024-03-29T04:11:30.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.556+0000 7fe9dc6e4640 1 -- start start 2024-03-29T04:11:30.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9dc6e4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9d4144a80 0x7fe9d4071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9dc6e4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe9d40d05b0 0x7fe9d40729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9dc6e4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d40b2fc0 con 0x7fe9d40720b0 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9dc6e4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d40b32a0 con 0x7fe9d4144a80 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9dc6e4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d40b3580 con 0x7fe9d40d05b0 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9da480640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9d4144a80 0x7fe9d4071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9d9c7f640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe9d40720b0 0x7fe9d40cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38134/0 (socket says 172.21.15.181:38134) 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9d9c7f640 1 -- 172.21.15.181:0/495940593 learned_addr learned my addr 172.21.15.181:0/495940593 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9da480640 1 -- 172.21.15.181:0/495940593 >> v2:172.21.15.187:3300/0 conn(0x7fe9d40d05b0 msgr2=0x7fe9d40729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9da480640 1 --2- 172.21.15.181:0/495940593 >> v2:172.21.15.187:3300/0 conn(0x7fe9d40d05b0 0x7fe9d40729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9da480640 1 -- 172.21.15.181:0/495940593 >> v1:172.21.15.181:6789/0 conn(0x7fe9d40720b0 legacy=0x7fe9d40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9da480640 1 -- 172.21.15.181:0/495940593 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d40b3860 con 0x7fe9d4144a80 2024-03-29T04:11:30.537 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.557+0000 7fe9d947e640 1 -- 172.21.15.181:0/495940593 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083548401 0 0) 0x7fe9d40b2fc0 con 0x7fe9d40720b0 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.558+0000 7fe9da480640 1 --2- 172.21.15.181:0/495940593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d4071b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40712b0 tx=0x7fe9d002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c470b153735456a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.558+0000 7fe9d947e640 1 -- 172.21.15.181:0/495940593 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00315a0 con 0x7fe9d4144a80 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.558+0000 7fe9d947e640 1 -- 172.21.15.181:0/495940593 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0037070 con 0x7fe9d4144a80 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.558+0000 7fe9d947e640 1 -- 172.21.15.181:0/495940593 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0031960 con 0x7fe9d4144a80 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/495940593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 msgr2=0x7fe9d4071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/495940593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d4071b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40712b0 tx=0x7fe9d002b840 comp rx=0 tx=0).stop 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/495940593 shutdown_connections 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/495940593 >> v2:172.21.15.187:3300/0 conn(0x7fe9d40d05b0 0x7fe9d40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/495940593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d4071b70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/495940593 >> 172.21.15.181:0/495940593 conn(0x7fe9d406f060 msgr2=0x7fe9d406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/495940593 shutdown_connections 2024-03-29T04:11:30.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.559+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/495940593 wait complete. 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.560+0000 7fe9dc6e4640 1 Processor -- start 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.560+0000 7fe9dc6e4640 1 -- start start 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d40720b0 0x7fe9d4143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 0x7fe9d415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4071620 con 0x7fe9d4144a80 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4144e60 con 0x7fe9d40720b0 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40717a0 con 0x7fe9d40d05b0 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60622/0 (socket says 172.21.15.181:60622) 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 -- 172.21.15.181:0/63842347 learned_addr learned my addr 172.21.15.181:0/63842347 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9da480640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d40720b0 0x7fe9d4143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9d9c7f640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 0x7fe9d415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 -- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 msgr2=0x7fe9d415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 0x7fe9d415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 -- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d40720b0 msgr2=0x7fe9d4143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d40720b0 0x7fe9d4143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 -- 172.21.15.181:0/63842347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d40af210 con 0x7fe9d4144a80 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9d9c7f640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 0x7fe9d415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dac81640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b9e0 tx=0x7fe9cc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc019070 con 0x7fe9d4144a80 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9cc00cb40 con 0x7fe9d4144a80 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc014440 con 0x7fe9d4144a80 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.561+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d40aef30 con 0x7fe9d4144a80 2024-03-29T04:11:30.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.562+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d414a750 con 0x7fe9d4144a80 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.562+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d4075900 con 0x7fe9d4144a80 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.563+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe9cc01c070 con 0x7fe9d4144a80 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.563+0000 7fe9caffd640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 0x7fe9b806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.563+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7fe9cc08b450 con 0x7fe9d4144a80 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.563+0000 7fe9da480640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 0x7fe9b806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:30.544 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.564+0000 7fe9da480640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 0x7fe9b806ea20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0036000 tx=0x7fe9d003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:30.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.568+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe9cc0592b0 con 0x7fe9d4144a80 2024-03-29T04:11:30.722 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:30.743+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9d40726f0 con 0x7fe9d4144a80 2024-03-29T04:11:31.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.450+0000 7fe9caffd640 1 -- 172.21.15.181:0/63842347 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7fe9cc05d5b0 con 0x7fe9d4144a80 2024-03-29T04:11:31.429 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_metadata' removed 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.454+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 msgr2=0x7fe9b806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.454+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 0x7fe9b806ea20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0036000 tx=0x7fe9d003d040 comp rx=0 tx=0).stop 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.454+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 msgr2=0x7fe9d415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.454+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b9e0 tx=0x7fe9cc00bea0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 shutdown_connections 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9d4144a80 0x7fe9d415cfd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe9b806c560 0x7fe9b806ea20 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0036000 tx=0x7fe9d003d040 comp rx=0 tx=0).stop 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9d40d05b0 0x7fe9d415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 --2- 172.21.15.181:0/63842347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9d40720b0 0x7fe9d4143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.455+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 >> 172.21.15.181:0/63842347 conn(0x7fe9d406f060 msgr2=0x7fe9d40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:31.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.456+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 shutdown_connections 2024-03-29T04:11:31.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.456+0000 7fe9dc6e4640 1 -- 172.21.15.181:0/63842347 wait complete. 2024-03-29T04:11:31.449 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T04:11:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 Processor -- start 2024-03-29T04:11:31.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 -- start start 2024-03-29T04:11:31.559 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e24158690 0x7f1e2406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 0x7f1e2406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e2415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2415eef0 con 0x7f1e24158690 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2415f1c0 con 0x7f1e240735a0 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.580+0000 7f1e28e1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e24072ac0 con 0x7f1e2415e500 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e24158690 0x7f1e2406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e24158690 0x7f1e2406d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60624/0 (socket says 172.21.15.181:60624) 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 -- 172.21.15.181:0/4065783102 learned_addr learned my addr 172.21.15.181:0/4065783102 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 -- 172.21.15.181:0/4065783102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 msgr2=0x7f1e2415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 --2- 172.21.15.181:0/4065783102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e2415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 -- 172.21.15.181:0/4065783102 >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 msgr2=0x7f1e2406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e1bfff640 1 --2- 172.21.15.181:0/4065783102 >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 0x7f1e2406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 --2- 172.21.15.181:0/4065783102 >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 0x7f1e2406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e2259c640 1 -- 172.21.15.181:0/4065783102 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e24072da0 con 0x7f1e24158690 2024-03-29T04:11:31.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.581+0000 7f1e1bfff640 1 --2- 172.21.15.181:0/4065783102 >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 0x7f1e2406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:31.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e2259c640 1 --2- 172.21.15.181:0/4065783102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e2406d220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1e240b2cb0 tx=0x7f1e1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d8d378d8e46617 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e21d9b640 1 -- 172.21.15.181:0/4065783102 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1402e070 con 0x7f1e24158690 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e21d9b640 1 -- 172.21.15.181:0/4065783102 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e14037070 con 0x7f1e24158690 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e21d9b640 1 -- 172.21.15.181:0/4065783102 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14031a00 con 0x7f1e24158690 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 -- 172.21.15.181:0/4065783102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 msgr2=0x7f1e2406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/4065783102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e2406d220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1e240b2cb0 tx=0x7f1e1402b880 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 -- 172.21.15.181:0/4065783102 shutdown_connections 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/4065783102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e2415e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/4065783102 >> v2:172.21.15.187:3300/0 conn(0x7f1e240735a0 0x7f1e2406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/4065783102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e2406d220 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 -- 172.21.15.181:0/4065783102 >> 172.21.15.181:0/4065783102 conn(0x7f1e2406f060 msgr2=0x7f1e2406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:31.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 -- 172.21.15.181:0/4065783102 shutdown_connections 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.582+0000 7f1e28e1b640 1 -- 172.21.15.181:0/4065783102 wait complete. 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 Processor -- start 2024-03-29T04:11:31.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 -- start start 2024-03-29T04:11:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e240735a0 0x7f1e241efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e241f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1e240cffd0 con 0x7f1e24158690 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e240cfe50 con 0x7f1e240735a0 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.584+0000 7f1e28e1b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e240d0150 con 0x7f1e2415e500 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60632/0 (socket says 172.21.15.181:60632) 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 -- 172.21.15.181:0/2724246160 learned_addr learned my addr 172.21.15.181:0/2724246160 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e2259c640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e240735a0 0x7f1e241efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e22d9d640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e241f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 -- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 msgr2=0x7f1e241f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e241f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 -- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e240735a0 msgr2=0x7f1e241efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e240735a0 0x7f1e241efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:31.566 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 -- 172.21.15.181:0/2724246160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e24155200 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1bfff640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c7f0 tx=0x7f1e0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c015070 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c007a30 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241f05f0 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c007e00 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.585+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241f0bb0 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.586+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e24075900 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.587+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1e0c018070 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.587+0000 7f1e1affd640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 0x7f1e0406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.587+0000 7f1e2259c640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 0x7f1e0406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.587+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5778+0+0 (secure 0 0 0) 0x7f1e0c08b700 con 0x7f1e24158690 2024-03-29T04:11:31.567 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.588+0000 7f1e2259c640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 0x7f1e0406e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e14036000 tx=0x7f1e1403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:31.570 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.592+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1e0c059cd0 con 0x7f1e24158690 2024-03-29T04:11:31.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:31.767+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e240cfae0 con 0x7f1e24158690 2024-03-29T04:11:32.433 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.454+0000 7f1e1affd640 1 -- 172.21.15.181:0/2724246160 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7f1e0c0595d0 con 0x7f1e24158690 2024-03-29T04:11:32.433 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'cephfs_data' removed 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 msgr2=0x7f1e0406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 0x7f1e0406e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1e14036000 tx=0x7f1e1403d040 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 msgr2=0x7f1e24154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00c7f0 tx=0x7f1e0c00cbf0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 shutdown_connections 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f1e0406c530 0x7f1e0406e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e2415e500 0x7f1e241f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e24158690 0x7f1e24154cc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 --2- 172.21.15.181:0/2724246160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e240735a0 0x7f1e241efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.456+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 >> 172.21.15.181:0/2724246160 conn(0x7f1e2406f060 msgr2=0x7f1e240cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.457+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 shutdown_connections 2024-03-29T04:11:32.436 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.457+0000 7f1e28e1b640 1 -- 172.21.15.181:0/2724246160 wait complete. 2024-03-29T04:11:32.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T04:11:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.574+0000 7fe23155f640 1 Processor -- start 2024-03-29T04:11:32.554 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 -- start start 2024-03-29T04:11:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe22c073310 0x7fe22c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22c1630b0 con 0x7fe22c0d1be0 2024-03-29T04:11:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe22c073880 con 0x7fe22c073310 2024-03-29T04:11:32.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.575+0000 7fe23155f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0d0e20 con 0x7fe22c15e5b0 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe22c073310 0x7fe22c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22affd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:44578/0 (socket says 172.21.15.181:44578) 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe22c073310 0x7fe22c06b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:48972/0 (socket says 172.21.15.181:48972) 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22affd640 1 -- 172.21.15.181:0/834649231 learned_addr learned my addr 172.21.15.181:0/834649231 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22b7fe640 1 --1- 172.21.15.181:0/834649231 >> v1:172.21.15.187:6789/0 conn(0x7fe22c0d1be0 0x7fe22c0b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:38966/0 (socket says 172.21.15.181:38966) 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 -- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 msgr2=0x7fe22c162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 -- 172.21.15.181:0/834649231 >> v1:172.21.15.187:6789/0 conn(0x7fe22c0d1be0 legacy=0x7fe22c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22a7fc640 1 -- 172.21.15.181:0/834649231 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c0d1100 con 0x7fe22c073310 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe229ffb640 1 -- 172.21.15.181:0/834649231 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210983158 0 0) 0x7fe22c1630b0 con 0x7fe22c0d1be0 2024-03-29T04:11:32.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.576+0000 7fe22affd640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c073310 0x7fe22c06b580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2140097a0 tx=0x7fe21402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87537836d4a6f99b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe229ffb640 1 -- 172.21.15.181:0/834649231 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140315a0 con 0x7fe22c073310 2024-03-29T04:11:32.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe229ffb640 1 -- 172.21.15.181:0/834649231 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe214037070 con 0x7fe22c073310 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe229ffb640 1 -- 172.21.15.181:0/834649231 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214031960 con 0x7fe22c073310 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe23155f640 1 -- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c073310 msgr2=0x7fe22c06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.577+0000 7fe23155f640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c073310 0x7fe22c06b580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2140097a0 tx=0x7fe21402b840 comp rx=0 tx=0).stop 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 -- 172.21.15.181:0/834649231 shutdown_connections 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c073310 0x7fe22c06b580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 --2- 172.21.15.181:0/834649231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 -- 172.21.15.181:0/834649231 >> 172.21.15.181:0/834649231 conn(0x7fe22c06e720 msgr2=0x7fe22c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 -- 172.21.15.181:0/834649231 shutdown_connections 2024-03-29T04:11:32.558 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.578+0000 7fe23155f640 1 -- 172.21.15.181:0/834649231 wait complete. 2024-03-29T04:11:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.579+0000 7fe23155f640 1 Processor -- start 2024-03-29T04:11:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.579+0000 7fe23155f640 1 -- start start 2024-03-29T04:11:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 0x7fe22c156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c15e990 con 0x7fe22c073310 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c0d16c0 con 0x7fe22c0d1be0 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c06bac0 con 0x7fe22c15e5b0 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60644/0 (socket says 172.21.15.181:60644) 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 -- 172.21.15.181:0/1140565522 learned_addr learned my addr 172.21.15.181:0/1140565522 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22b7fe640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 0x7fe22c156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 -- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 msgr2=0x7fe22c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 -- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 msgr2=0x7fe22c156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 0x7fe22c156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 -- 172.21.15.181:0/1140565522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c14e020 con 0x7fe22c073310 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 0x7fe22c156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:32.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe22affd640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009e80 tx=0x7fe21c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00ce60 con 0x7fe22c073310 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c019070 con 0x7fe22c073310 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c014680 con 0x7fe22c073310 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c155d30 con 0x7fe22c073310 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.580+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c0b0e60 con 0x7fe22c073310 2024-03-29T04:11:32.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.581+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c075890 con 0x7fe22c073310 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.582+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe21c005d10 con 0x7fe22c073310 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.582+0000 7fe2237fe640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 0x7fe20c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.582+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe21c08b9f0 con 0x7fe22c073310 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.582+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 0x7fe20c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.563 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.583+0000 7fe22a7fc640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 0x7fe20c06ea20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe214036000 tx=0x7fe214008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.565 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.586+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe22c14e020 con 0x7fe22c073310 2024-03-29T04:11:32.743 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.763+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe22c06be90 con 0x7fe22c073310 2024-03-29T04:11:32.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.767+0000 7fe2237fe640 1 -- 172.21.15.181:0/1140565522 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fe21c010040 con 0x7fe22c073310 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 msgr2=0x7fe20c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 0x7fe20c06ea20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe214036000 tx=0x7fe214008030 comp rx=0 tx=0).stop 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 msgr2=0x7fe22c15eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009e80 tx=0x7fe21c00c490 comp rx=0 tx=0).stop 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 shutdown_connections 2024-03-29T04:11:32.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe20c06c560 0x7fe20c06ea20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe22c15e5b0 0x7fe22c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe22c0d1be0 0x7fe22c156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 --2- 172.21.15.181:0/1140565522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe22c073310 0x7fe22c15eb10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 >> 172.21.15.181:0/1140565522 conn(0x7fe22c06e720 msgr2=0x7fe22c0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 shutdown_connections 2024-03-29T04:11:32.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.769+0000 7fe23155f640 1 -- 172.21.15.181:0/1140565522 wait complete. 2024-03-29T04:11:32.759 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-29T04:11:32.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.880+0000 7f509dbcd640 1 Processor -- start 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.880+0000 7f509dbcd640 1 -- start start 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f509dbcd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f509815e950 0x7f50980d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f509dbcd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50980d48c0 0x7f50980d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f509dbcd640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50980d36f0 con 0x7f50980d2f00 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f509dbcd640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f50980d3cb0 con 0x7f509815e950 2024-03-29T04:11:32.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f509dbcd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50980d4d00 con 0x7f50980d48c0 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50977fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f509815e950 0x7f50980d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f5097fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f50980d2f00 0x7f50980cef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38140/0 (socket says 172.21.15.181:38140) 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f5097fff640 1 -- 172.21.15.181:0/1322600063 learned_addr learned my addr 172.21.15.181:0/1322600063 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f5096ffd640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50980d48c0 0x7f50980d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50977fe640 1 -- 172.21.15.181:0/1322600063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50980d48c0 msgr2=0x7f50980d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50977fe640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50980d48c0 0x7f50980d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50967fc640 1 -- 172.21.15.181:0/1322600063 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945787176 0 0) 0x7f50980d36f0 con 0x7f50980d2f00 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50977fe640 1 -- 172.21.15.181:0/1322600063 >> v1:172.21.15.181:6789/0 conn(0x7f50980d2f00 legacy=0x7f50980cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.881+0000 7f50977fe640 1 -- 172.21.15.181:0/1322600063 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50980d39d0 con 0x7f509815e950 2024-03-29T04:11:32.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.882+0000 7f50977fe640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f509815e950 0x7f50980d4380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50880097a0 tx=0x7f508802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274fdbf93aa6ac99 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.882+0000 7f50967fc640 1 -- 172.21.15.181:0/1322600063 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088034070 con 0x7f509815e950 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.882+0000 7f50967fc640 1 -- 172.21.15.181:0/1322600063 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5088030610 con 0x7f509815e950 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.882+0000 7f50967fc640 1 -- 172.21.15.181:0/1322600063 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880309e0 con 0x7f509815e950 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 -- 172.21.15.181:0/1322600063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f509815e950 msgr2=0x7f50980d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f509815e950 0x7f50980d4380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50880097a0 tx=0x7f508802b960 comp rx=0 tx=0).stop 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 -- 172.21.15.181:0/1322600063 shutdown_connections 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50980d48c0 0x7f50980d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 --2- 172.21.15.181:0/1322600063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f509815e950 0x7f50980d4380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 -- 172.21.15.181:0/1322600063 >> 172.21.15.181:0/1322600063 conn(0x7f50980676e0 msgr2=0x7f5098067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 -- 172.21.15.181:0/1322600063 shutdown_connections 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.883+0000 7f509dbcd640 1 -- 172.21.15.181:0/1322600063 wait complete. 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.884+0000 7f509dbcd640 1 Processor -- start 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.884+0000 7f509dbcd640 1 -- start start 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50980d2f00 0x7f50980caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f509815e950 0x7f5098156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f50980d3390 con 0x7f50980d48c0 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50980b7da0 con 0x7f50980d2f00 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f509dbcd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f509815ed30 con 0x7f509815e950 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60656/0 (socket says 172.21.15.181:60656) 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 -- 172.21.15.181:0/3015510313 learned_addr learned my addr 172.21.15.181:0/3015510313 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5097fff640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f509815e950 0x7f5098156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f50977fe640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50980d2f00 0x7f50980caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 -- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f509815e950 msgr2=0x7f5098156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f509815e950 0x7f5098156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 -- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50980d2f00 msgr2=0x7f50980caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50980d2f00 0x7f50980caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 -- 172.21.15.181:0/3015510313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50980caf80 con 0x7f50980d48c0 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.885+0000 7f5096ffd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f508c00e9b0 tx=0x7f508c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.886+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c017070 con 0x7f50980d48c0 2024-03-29T04:11:32.873 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.886+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c00c9a0 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.886+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5098160d10 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.886+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c00cd90 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.886+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981612d0 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.887+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5098075a10 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.887+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f508c01a070 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.887+0000 7f507ffff640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 0x7f508406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.888+0000 7f50977fe640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 0x7f508406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.888+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f508c08b440 con 0x7f50980d48c0 2024-03-29T04:11:32.874 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.888+0000 7f50977fe640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 0x7f508406e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5088036000 tx=0x7f508803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:32.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:32.892+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f508c059610 con 0x7f50980d48c0 2024-03-29T04:11:33.131 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.152+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f50980cfdf0 con 0x7f50980d48c0 2024-03-29T04:11:33.434 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.455+0000 7f507ffff640 1 -- 172.21.15.181:0/3015510313 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v140) ==== 147+0+0 (secure 0 0 0) 0x7f508c05d910 con 0x7f50980d48c0 2024-03-29T04:11:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 msgr2=0x7f508406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 0x7f508406e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5088036000 tx=0x7f508803b040 comp rx=0 tx=0).stop 2024-03-29T04:11:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 msgr2=0x7f5098160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f508c00e9b0 tx=0x7f508c00ee70 comp rx=0 tx=0).stop 2024-03-29T04:11:33.438 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 shutdown_connections 2024-03-29T04:11:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f508406c530 0x7f508406e9f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f509815e950 0x7f5098156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980d48c0 0x7f5098160330 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 --2- 172.21.15.181:0/3015510313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50980d2f00 0x7f50980caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:33.439 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.457+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 >> 172.21.15.181:0/3015510313 conn(0x7f50980676e0 msgr2=0x7f50980cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.458+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 shutdown_connections 2024-03-29T04:11:33.440 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:33.458+0000 7f509dbcd640 1 -- 172.21.15.181:0/3015510313 wait complete. 2024-03-29T04:11:33.452 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-29T04:11:33.453 INFO:tasks.cephfs_test_runner: 2024-03-29T04:11:33.453 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:11:33.453 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:11:33.453 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:11:33.454 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:11:33.455 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:11:33.456 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:11:33.457 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner: 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner:Ran 1 test in 96.335s 2024-03-29T04:11:33.458 INFO:tasks.cephfs_test_runner: 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner: 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T04:11:33.459 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 259, in run_test_worker 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: info = self.get_inline_data_info(inodes, files, dir_pins, num_mds) 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 116, in get_inline_data_info 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: json_out = get_inode_dump(inode, 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/test_uninlining.py", line 101, in get_inode_dump 2024-03-29T04:11:33.460 INFO:tasks.cephfs_test_runner: json_out = self.fs.rank_tell(['dump', 'inode', inode], rank) 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-29T04:11:33.461 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-29T04:11:33.462 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-29T04:11:33.463 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-29T04:11:33.464 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-29T04:11:33.464 INFO:tasks.cephfs_test_runner: 2024-03-29T04:11:33.464 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:11:33.713 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c7d73a7385aa414093fa018315f76f52 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_69e2cd73c52284dfd5096c66b60c6e6b26a3c447/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-29T04:11:33.715 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T04:11:33.745 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-29T04:11:33.760 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-29T04:11:33.760 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:11:33.761 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:33.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056349,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15338 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:11:33.781 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-03-29T04:11:33.781 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:33.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056349,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15338 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T04:11:33.838 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:11:33.838 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:33.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2267 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:11:33.857 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-03-29T04:11:33.857 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T04:11:33.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2267 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T04:11:33.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:33.916 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:11:33.934 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T04:11:33.934 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T04:11:33.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:33.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T04:11:34.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T04:11:34.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-03-29T04:11:34.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.170 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-03-29T04:11:34.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-03-29T04:11:34.255 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2024-03-29T04:11:34.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:11:34.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.256 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:11:34.275 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T04:11:34.275 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T04:11:34.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.275 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T04:11:34.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.346 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T04:11:34.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.416 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-03-29T04:11:34.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.500 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-03-29T04:11:34.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T04:11:34.556 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-03-29T04:11:34.630 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-03-29T04:11:34.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:11:34.631 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T04:11:34.645 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T04:11:34.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:34.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.767+0000 7fb66f959640 1 Processor -- start 2024-03-29T04:11:34.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.768+0000 7fb66f959640 1 -- start start 2024-03-29T04:11:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.768+0000 7fb66f959640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb6680cbda0 0x7fb66815f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.768+0000 7fb66f959640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb668144e10 con 0x7fb6681445e0 2024-03-29T04:11:34.751 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.768+0000 7fb66f959640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6680caa80 con 0x7fb6680cf070 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.768+0000 7fb66f959640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb6680cd610 con 0x7fb6680cbda0 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66d6f5640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb6681445e0 0x7fb668072340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38152/0 (socket says 172.21.15.181:38152) 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66d6f5640 1 -- 172.21.15.181:0/1135474503 learned_addr learned my addr 172.21.15.181:0/1135474503 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2957351103 0 0) 0x7fb668144e10 con 0x7fb6681445e0 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003520 con 0x7fb6681445e0 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1615632264 0 0) 0x7fb654003520 con 0x7fb6681445e0 2024-03-29T04:11:34.752 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 >> v2:172.21.15.187:3300/0 conn(0x7fb6680cbda0 msgr2=0x7fb66815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 --2- 172.21.15.181:0/1135474503 >> v2:172.21.15.187:3300/0 conn(0x7fb6680cbda0 0x7fb66815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 >> v1:172.21.15.187:6789/0 conn(0x7fb6680cf070 legacy=0x7fb6680cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680cad60 con 0x7fb6681445e0 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fb658001670 con 0x7fb6681445e0 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 >> v1:172.21.15.181:6789/0 conn(0x7fb6681445e0 legacy=0x7fb668072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6540039b0 0x7fb654003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb654004800 0x7fb654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.753 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb65400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680cd610 con 0x7fb6540052e0 2024-03-29T04:11:34.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb65400c1e0 con 0x7fb654004800 2024-03-29T04:11:34.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb65400e5f0 con 0x7fb6540039b0 2024-03-29T04:11:34.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb65400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66cef4640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb654004800 0x7fb654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66d6f5640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6540039b0 0x7fb654003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 -- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6540039b0 msgr2=0x7fb654003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6540039b0 0x7fb654003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 -- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb654004800 msgr2=0x7fb654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb654004800 0x7fb654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.755 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66def6640 1 -- 172.21.15.181:0/1135474503 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb654003520 con 0x7fb6540052e0 2024-03-29T04:11:34.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66cef4640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb654004800 0x7fb654004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:34.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.769+0000 7fb66d6f5640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6540039b0 0x7fb654003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:34.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.770+0000 7fb66def6640 1 -- 172.21.15.181:0/1135474503 reap_dead start 2024-03-29T04:11:34.756 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.770+0000 7fb66def6640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb65400b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb660009e10 tx=0x7fb654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5a2e74ebd9bacb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.770+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660009a10 con 0x7fb6540052e0 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.770+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb660006e50 con 0x7fb6540052e0 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.770+0000 7fb65ffff640 1 -- 172.21.15.181:0/1135474503 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660003970 con 0x7fb6540052e0 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 -- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 msgr2=0x7fb65400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb65400b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb660009e10 tx=0x7fb654005830 comp rx=0 tx=0).stop 2024-03-29T04:11:34.757 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 -- 172.21.15.181:0/1135474503 shutdown_connections 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 --2- 172.21.15.181:0/1135474503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb65400b9c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 -- 172.21.15.181:0/1135474503 >> 172.21.15.181:0/1135474503 conn(0x7fb6680b1c90 msgr2=0x7fb6680b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 -- 172.21.15.181:0/1135474503 shutdown_connections 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.771+0000 7fb66f959640 1 -- 172.21.15.181:0/1135474503 wait complete. 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.772+0000 7fb66f959640 1 Processor -- start 2024-03-29T04:11:34.758 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.772+0000 7fb66f959640 1 -- start start 2024-03-29T04:11:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb668142960 0x7fb668142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.761 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6681584f0 0x7fb6681589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb66815edb0 con 0x7fb6540052e0 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681449c0 con 0x7fb668142960 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66f959640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680cb320 con 0x7fb6681584f0 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60678/0 (socket says 172.21.15.181:60678) 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 -- 172.21.15.181:0/2517601555 learned_addr learned my addr 172.21.15.181:0/2517601555 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:34.762 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66cef4640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb668142960 0x7fb668142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66def6640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6681584f0 0x7fb6681589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 -- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6681584f0 msgr2=0x7fb6681589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6681584f0 0x7fb6681589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 -- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb668142960 msgr2=0x7fb668142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb668142960 0x7fb668142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 -- 172.21.15.181:0/2517601555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6681418c0 con 0x7fb6540052e0 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb66d6f5640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb65800e820 tx=0x7fb65800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.763 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65800c870 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.773+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658018070 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.774+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb668141ba0 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.774+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65800cc30 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.774+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb668142160 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.775+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb668075a10 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.775+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb658013420 con 0x7fb6540052e0 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.775+0000 7fb65dffb640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 0x7fb64c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:34.764 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.776+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb65808af60 con 0x7fb6540052e0 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.776+0000 7fb66cef4640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 0x7fb64c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.776+0000 7fb66cef4640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 0x7fb64c06ea20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb664007fd0 tx=0x7fb664009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:34.765 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.780+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb6580590a0 con 0x7fb6540052e0 2024-03-29T04:11:34.930 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.951+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6680cd790 con 0x7fb64c06c560 2024-03-29T04:11:34.931 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.952+0000 7fb65dffb640 1 -- 172.21.15.181:0/2517601555 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13561 (secure 0 0 0) 0x7fb6680cd790 con 0x7fb64c06c560 2024-03-29T04:11:34.931 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:34.932 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:11:34.933 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 msgr2=0x7fb64c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 0x7fb64c06ea20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb664007fd0 tx=0x7fb664009040 comp rx=0 tx=0).stop 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 msgr2=0x7fb668140370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb65800e820 tx=0x7fb65800ece0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 shutdown_connections 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb64c06c560 0x7fb64c06ea20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6681584f0 0x7fb6681589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.934 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb668142960 0x7fb668142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 --2- 172.21.15.181:0/2517601555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6540052e0 0x7fb668140370 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 >> 172.21.15.181:0/2517601555 conn(0x7fb6680b1c90 msgr2=0x7fb6680af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 shutdown_connections 2024-03-29T04:11:34.936 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:34.954+0000 7fb66f959640 1 -- 172.21.15.181:0/2517601555 wait complete. 2024-03-29T04:11:34.944 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-29T04:11:33.599377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":566576,"kb_used_data":346280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754408144,"statfs":{"total":773094113280,"available":772513939456,"internally_reserved":0,"allocated":354590720,"data_stored":352020510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174415068,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-350,"num_write_kb":-170472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":210,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-29T04:10:58.399122+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:58.399122+0000","last_peered":"2024-03-29T04:10:58.399122+0000","last_clean":"2024-03-29T04:10:58.399122+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:58.399122+0000","last_undegraded":"2024-03-29T04:10:58.399122+0000","last_fullsized":"2024-03-29T04:10:58.399122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94644,"kb_used_data":63468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277196,"statfs":{"total":96636764160,"available":96539848704,"internally_reserved":0,"allocated":64991232,"data_stored":64671025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69436,"kb_used_data":42420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302404,"statfs":{"total":96636764160,"available":96565661696,"internally_reserved":0,"allocated":43438080,"data_stored":43115709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97548,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94274292,"statfs":{"total":96636764160,"available":96536875008,"internally_reserved":0,"allocated":72220672,"data_stored":71887471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44328,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327512,"statfs":{"total":96636764160,"available":96591372288,"internally_reserved":0,"allocated":17723392,"data_stored":17402071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82328,"kb_used_data":55312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289512,"statfs":{"total":96636764160,"available":96552460288,"internally_reserved":0,"allocated":56639488,"data_stored":56294346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77152,"kb_used_data":50136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294688,"statfs":{"total":96636764160,"available":96557760512,"internally_reserved":0,"allocated":51339264,"data_stored":51031546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":12976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":13287424,"data_stored":12995037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61148,"kb_used_data":34132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310692,"statfs":{"total":96636764160,"available":96574148608,"internally_reserved":0,"allocated":34951168,"data_stored":34623305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:34.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:35.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.066+0000 7fe6474d9640 1 Processor -- start 2024-03-29T04:11:35.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.066+0000 7fe6474d9640 1 -- start start 2024-03-29T04:11:35.048 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe6474d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe6401672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe6474d9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe640146040 con 0x7fe640071c30 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe6474d9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe640160e50 con 0x7fe640145c60 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe6474d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe64015d8b0 con 0x7fe640160830 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe644a74640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe640071c30 0x7fe640160120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38160/0 (socket says 172.21.15.181:38160) 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe644a74640 1 -- 172.21.15.181:0/2789198574 learned_addr learned my addr 172.21.15.181:0/2789198574 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe645a76640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe6401672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.049 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226337858 0 0) 0x7fe640146040 con 0x7fe640071c30 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe62c003520 con 0x7fe640071c30 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 396247622 0 0) 0x7fe62c003520 con 0x7fe640071c30 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 msgr2=0x7fe6401672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe6401672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 >> v1:172.21.15.187:6789/0 conn(0x7fe640145c60 legacy=0x7fe640071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64015caf0 con 0x7fe640071c30 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7fe630001670 con 0x7fe640071c30 2024-03-29T04:11:35.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 >> v1:172.21.15.181:6789/0 conn(0x7fe640071c30 legacy=0x7fe640160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c0039e0 0x7fe62c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe62c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe64015d8b0 con 0x7fe62c0039e0 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c00baf0 con 0x7fe62c004800 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c00bcf0 con 0x7fe640160830 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.067+0000 7fe645275640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c0039e0 0x7fe62c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe644a74640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645a76640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe62c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 -- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 msgr2=0x7fe62c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe640160830 0x7fe62c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 -- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 msgr2=0x7fe62c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c004800 0x7fe62c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 -- 172.21.15.181:0/2789198574 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c003520 con 0x7fe62c0039e0 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.068+0000 7fe645275640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c0039e0 0x7fe62c003e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe63c001750 tx=0x7fe62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84409787c83876de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c001370 con 0x7fe62c0039e0 2024-03-29T04:11:35.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c0052d0 con 0x7fe62c0039e0 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe637fff640 1 -- 172.21.15.181:0/2789198574 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003830 con 0x7fe62c0039e0 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 -- 172.21.15.181:0/2789198574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c0039e0 msgr2=0x7fe62c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 --2- 172.21.15.181:0/2789198574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62c0039e0 0x7fe62c003e00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe63c001750 tx=0x7fe62c005830 comp rx=0 tx=0).stop 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe645a76640 1 -- 172.21.15.181:0/2789198574 reap_dead start 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 -- 172.21.15.181:0/2789198574 shutdown_connections 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 -- 172.21.15.181:0/2789198574 >> 172.21.15.181:0/2789198574 conn(0x7fe6400b1c90 msgr2=0x7fe6400b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 -- 172.21.15.181:0/2789198574 shutdown_connections 2024-03-29T04:11:35.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.069+0000 7fe6474d9640 1 -- 172.21.15.181:0/2789198574 wait complete. 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.070+0000 7fe6474d9640 1 Processor -- start 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.070+0000 7fe6474d9640 1 -- start start 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 0x7fe64013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64013b8e0 0x7fe64015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6400cb840 con 0x7fe64013aa20 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe64015d0b0 con 0x7fe62c0039e0 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe6474d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6401652e0 con 0x7fe64013b8e0 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60702/0 (socket says 172.21.15.181:60702) 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 -- 172.21.15.181:0/1898289591 learned_addr learned my addr 172.21.15.181:0/1898289591 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645275640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 0x7fe64013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe644a74640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64013b8e0 0x7fe64015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.057 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 -- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64013b8e0 msgr2=0x7fe64015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64013b8e0 0x7fe64015b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 -- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 msgr2=0x7fe64013d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 0x7fe64013d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645a76640 1 -- 172.21.15.181:0/1898289591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64013dfd0 con 0x7fe64013aa20 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.071+0000 7fe645275640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 0x7fe64013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe645a76640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe638011c80 tx=0x7fe63800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638015430 con 0x7fe64013aa20 2024-03-29T04:11:35.058 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6380155d0 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6400cdcc0 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380159c0 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.072+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64013dcf0 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.073+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe640075a10 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.073+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe638015b60 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.074+0000 7fe635ffb640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 0x7fe62406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.074+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe63808c300 con 0x7fe64013aa20 2024-03-29T04:11:35.059 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.074+0000 7fe645275640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 0x7fe62406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.074+0000 7fe645275640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 0x7fe62406e9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe63c033000 tx=0x7fe63c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.060 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.078+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe638019030 con 0x7fe64013aa20 2024-03-29T04:11:35.230 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.249+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6400ca610 con 0x7fe62406c530 2024-03-29T04:11:35.231 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.250+0000 7fe635ffb640 1 -- 172.21.15.181:0/1898289591 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13561 (secure 0 0 0) 0x7fe6400ca610 con 0x7fe62406c530 2024-03-29T04:11:35.231 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:11:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 msgr2=0x7fe62406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 0x7fe62406e9f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe63c033000 tx=0x7fe63c009030 comp rx=0 tx=0).stop 2024-03-29T04:11:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 msgr2=0x7fe64013aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe638011c80 tx=0x7fe63800f580 comp rx=0 tx=0).stop 2024-03-29T04:11:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 shutdown_connections 2024-03-29T04:11:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe64013aa20 0x7fe64013aea0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.235 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe62406c530 0x7fe62406e9f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64013b8e0 0x7fe64015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 --2- 172.21.15.181:0/1898289591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe62c0039e0 0x7fe64013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 >> 172.21.15.181:0/1898289591 conn(0x7fe6400b1c90 msgr2=0x7fe6400ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.236 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 shutdown_connections 2024-03-29T04:11:35.237 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.252+0000 7fe6474d9640 1 -- 172.21.15.181:0/1898289591 wait complete. 2024-03-29T04:11:35.242 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-29T04:11:33.599377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":566576,"kb_used_data":346280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754408144,"statfs":{"total":773094113280,"available":772513939456,"internally_reserved":0,"allocated":354590720,"data_stored":352020510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174415068,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-350,"num_write_kb":-170472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":210,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-29T04:10:58.399122+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:58.399122+0000","last_peered":"2024-03-29T04:10:58.399122+0000","last_clean":"2024-03-29T04:10:58.399122+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:58.399122+0000","last_undegraded":"2024-03-29T04:10:58.399122+0000","last_fullsized":"2024-03-29T04:10:58.399122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94644,"kb_used_data":63468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277196,"statfs":{"total":96636764160,"available":96539848704,"internally_reserved":0,"allocated":64991232,"data_stored":64671025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69436,"kb_used_data":42420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302404,"statfs":{"total":96636764160,"available":96565661696,"internally_reserved":0,"allocated":43438080,"data_stored":43115709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97548,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94274292,"statfs":{"total":96636764160,"available":96536875008,"internally_reserved":0,"allocated":72220672,"data_stored":71887471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44328,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327512,"statfs":{"total":96636764160,"available":96591372288,"internally_reserved":0,"allocated":17723392,"data_stored":17402071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82328,"kb_used_data":55312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289512,"statfs":{"total":96636764160,"available":96552460288,"internally_reserved":0,"allocated":56639488,"data_stored":56294346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77152,"kb_used_data":50136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294688,"statfs":{"total":96636764160,"available":96557760512,"internally_reserved":0,"allocated":51339264,"data_stored":51031546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":12976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":13287424,"data_stored":12995037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61148,"kb_used_data":34132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310692,"statfs":{"total":96636764160,"available":96574148608,"internally_reserved":0,"allocated":34951168,"data_stored":34623305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:35.243 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T04:11:35.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.364+0000 7ff2faf85640 1 Processor -- start 2024-03-29T04:11:35.345 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2faf85640 1 -- start start 2024-03-29T04:11:35.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2faf85640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2f415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2faf85640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f415b650 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2faf85640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f4072ab0 con 0x7ff2f4072280 2024-03-29T04:11:35.351 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2faf85640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f4072d90 con 0x7ff2f415b270 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.365+0000 7ff2f9522640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff2f40cd0e0 0x7ff2f40ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38176/0 (socket says 172.21.15.181:38176) 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f9522640 1 -- 172.21.15.181:0/2514294882 learned_addr learned my addr 172.21.15.181:0/2514294882 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586233763 0 0) 0x7ff2f415b650 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2e4003520 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f8d21640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2f415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396601984 0 0) 0x7ff2e4003520 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.352 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 msgr2=0x7ff2f415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2f415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 >> v1:172.21.15.187:6789/0 conn(0x7ff2f4072280 legacy=0x7ff2f40cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f4073070 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7ff2ec0019f0 con 0x7ff2f40cd0e0 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 >> v1:172.21.15.181:6789/0 conn(0x7ff2f40cd0e0 legacy=0x7ff2f40ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2e40039e0 0x7ff2e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.353 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4072d90 con 0x7ff2e4004800 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e400ba70 con 0x7ff2e40039e0 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e400bc70 con 0x7ff2f415b270 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f9522640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f8d21640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2e40039e0 0x7ff2e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.354 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 -- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 msgr2=0x7ff2e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f415b270 0x7ff2e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 -- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2e40039e0 msgr2=0x7ff2e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2e40039e0 0x7ff2e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.366+0000 7ff2f3fff640 1 -- 172.21.15.181:0/2514294882 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e4003520 con 0x7ff2e4004800 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.367+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2e4004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0001720 tx=0x7ff2e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfacd9c91b6c86ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.367+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0001370 con 0x7ff2e4004800 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.367+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e00052d0 con 0x7ff2e4004800 2024-03-29T04:11:35.355 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.367+0000 7ff2f37fe640 1 -- 172.21.15.181:0/2514294882 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0003830 con 0x7ff2e4004800 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 -- 172.21.15.181:0/2514294882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 msgr2=0x7ff2e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2514294882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2e4004c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0001720 tx=0x7ff2e4005830 comp rx=0 tx=0).stop 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2f9522640 1 -- 172.21.15.181:0/2514294882 reap_dead start 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 -- 172.21.15.181:0/2514294882 shutdown_connections 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 -- 172.21.15.181:0/2514294882 >> 172.21.15.181:0/2514294882 conn(0x7ff2f40b1c90 msgr2=0x7ff2f40b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 -- 172.21.15.181:0/2514294882 shutdown_connections 2024-03-29T04:11:35.356 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.368+0000 7ff2faf85640 1 -- 172.21.15.181:0/2514294882 wait complete. 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.369+0000 7ff2faf85640 1 Processor -- start 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.369+0000 7ff2faf85640 1 -- start start 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f41e86e0 0x7ff2f4158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2f40c7310 0x7ff2f40c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f40726e0 con 0x7ff2e4004800 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4073630 con 0x7ff2f40c7310 2024-03-29T04:11:35.357 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2faf85640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f4072860 con 0x7ff2f41e86e0 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60716/0 (socket says 172.21.15.181:60716) 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 -- 172.21.15.181:0/2558999598 learned_addr learned my addr 172.21.15.181:0/2558999598 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f41e86e0 0x7ff2f4158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f9522640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2f40c7310 0x7ff2f40c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 -- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f41e86e0 msgr2=0x7ff2f4158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.358 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f41e86e0 0x7ff2f4158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 -- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2f40c7310 msgr2=0x7ff2f40c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2f40c7310 0x7ff2f40c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 -- 172.21.15.181:0/2558999598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f40a9490 con 0x7ff2e4004800 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.370+0000 7ff2f8d21640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80119d0 tx=0x7ff2e8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.371+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800fb50 con 0x7ff2e4004800 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.371+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e800fcf0 con 0x7ff2e4004800 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.371+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800fea0 con 0x7ff2e4004800 2024-03-29T04:11:35.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.371+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f41e79e0 con 0x7ff2e4004800 2024-03-29T04:11:35.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.371+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41e7fa0 con 0x7ff2e4004800 2024-03-29T04:11:35.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.372+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f4075a10 con 0x7ff2e4004800 2024-03-29T04:11:35.360 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.372+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff2e8012040 con 0x7ff2e4004800 2024-03-29T04:11:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.372+0000 7ff2f17fa640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 0x7ff2d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.373+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7ff2e808bc50 con 0x7ff2e4004800 2024-03-29T04:11:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.373+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 0x7ff2d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.373+0000 7ff2f3fff640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 0x7ff2d806e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0033000 tx=0x7ff2e0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.361 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.377+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff2e8091020 con 0x7ff2e4004800 2024-03-29T04:11:35.529 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.550+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2f41e7790 con 0x7ff2d806c530 2024-03-29T04:11:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.551+0000 7ff2f17fa640 1 -- 172.21.15.181:0/2558999598 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13561 (secure 0 0 0) 0x7ff2f41e7790 con 0x7ff2d806c530 2024-03-29T04:11:35.530 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:35.530 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:11:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 msgr2=0x7ff2d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 0x7ff2d806e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0033000 tx=0x7ff2e0009030 comp rx=0 tx=0).stop 2024-03-29T04:11:35.532 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 msgr2=0x7ff2f41597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80119d0 tx=0x7ff2e8011e90 comp rx=0 tx=0).stop 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 shutdown_connections 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2f40c7310 0x7ff2f40c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7ff2d806c530 0x7ff2d806e9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2f41e86e0 0x7ff2f4158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 --2- 172.21.15.181:0/2558999598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2e4004800 0x7ff2f41597a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 >> 172.21.15.181:0/2558999598 conn(0x7ff2f40b1c90 msgr2=0x7ff2f40b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.533 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 shutdown_connections 2024-03-29T04:11:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.553+0000 7ff2faf85640 1 -- 172.21.15.181:0/2558999598 wait complete. 2024-03-29T04:11:35.543 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-29T04:11:33.599377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":566576,"kb_used_data":346280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754408144,"statfs":{"total":773094113280,"available":772513939456,"internally_reserved":0,"allocated":354590720,"data_stored":352020510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-174415068,"num_objects":-138,"num_object_clones":0,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":0,"num_read":-77,"num_read_kb":-2,"num_write":-350,"num_write_kb":-170472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":210,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-29T04:10:58.399122+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:10:58.399122+0000","last_peered":"2024-03-29T04:10:58.399122+0000","last_clean":"2024-03-29T04:10:58.399122+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:10:58.399122+0000","last_undegraded":"2024-03-29T04:10:58.399122+0000","last_fullsized":"2024-03-29T04:10:58.399122+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94644,"kb_used_data":63468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277196,"statfs":{"total":96636764160,"available":96539848704,"internally_reserved":0,"allocated":64991232,"data_stored":64671025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69436,"kb_used_data":42420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94302404,"statfs":{"total":96636764160,"available":96565661696,"internally_reserved":0,"allocated":43438080,"data_stored":43115709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97548,"kb_used_data":70528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94274292,"statfs":{"total":96636764160,"available":96536875008,"internally_reserved":0,"allocated":72220672,"data_stored":71887471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44328,"kb_used_data":17308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327512,"statfs":{"total":96636764160,"available":96591372288,"internally_reserved":0,"allocated":17723392,"data_stored":17402071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82328,"kb_used_data":55312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289512,"statfs":{"total":96636764160,"available":96552460288,"internally_reserved":0,"allocated":56639488,"data_stored":56294346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77152,"kb_used_data":50136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94294688,"statfs":{"total":96636764160,"available":96557760512,"internally_reserved":0,"allocated":51339264,"data_stored":51031546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39992,"kb_used_data":12976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331848,"statfs":{"total":96636764160,"available":96595812352,"internally_reserved":0,"allocated":13287424,"data_stored":12995037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61148,"kb_used_data":34132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310692,"statfs":{"total":96636764160,"available":96574148608,"internally_reserved":0,"allocated":34951168,"data_stored":34623305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:35.544 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T04:11:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.665+0000 7f0405f40640 1 Processor -- start 2024-03-29T04:11:35.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.665+0000 7f0405f40640 1 -- start start 2024-03-29T04:11:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f0405f40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400145c60 0x7f0400071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.646 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f0405f40640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0400160830 0x7f04000ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f0405f40640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0400146040 con 0x7f0400071a60 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f0405f40640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f0400071ea0 con 0x7f0400160830 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f0405f40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0400160e50 con 0x7f0400145c60 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f03fffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0400160830 0x7f04000ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f03fffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0400160830 0x7f04000ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60722/0 (socket says 172.21.15.181:60722) 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.666+0000 7f03fffff640 1 -- 172.21.15.181:0/439098556 learned_addr learned my addr 172.21.15.181:0/439098556 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fffff640 1 -- 172.21.15.181:0/439098556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400145c60 msgr2=0x7f0400071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.647 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fffff640 1 --2- 172.21.15.181:0/439098556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400145c60 0x7f0400071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fffff640 1 -- 172.21.15.181:0/439098556 >> v1:172.21.15.187:6789/0 conn(0x7f0400071a60 legacy=0x7f0400160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fffff640 1 -- 172.21.15.181:0/439098556 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040015d8b0 con 0x7f0400160830 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fffff640 1 --2- 172.21.15.181:0/439098556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400160830 0x7f04000ca830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009e50 tx=0x7f03f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827dfef031e2d2c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fe7fc640 1 -- 172.21.15.181:0/439098556 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f802c9e0 con 0x7f0400160830 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fe7fc640 1 -- 172.21.15.181:0/439098556 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f802cb80 con 0x7f0400160830 2024-03-29T04:11:35.648 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.667+0000 7f03fe7fc640 1 -- 172.21.15.181:0/439098556 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f80343f0 con 0x7f0400160830 2024-03-29T04:11:35.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 -- 172.21.15.181:0/439098556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400160830 msgr2=0x7f04000ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 --2- 172.21.15.181:0/439098556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400160830 0x7f04000ca830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009e50 tx=0x7f03f802be40 comp rx=0 tx=0).stop 2024-03-29T04:11:35.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 -- 172.21.15.181:0/439098556 shutdown_connections 2024-03-29T04:11:35.651 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 --2- 172.21.15.181:0/439098556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400160830 0x7f04000ca830 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 --2- 172.21.15.181:0/439098556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400145c60 0x7f0400071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 -- 172.21.15.181:0/439098556 >> 172.21.15.181:0/439098556 conn(0x7f04000b1c90 msgr2=0x7f04000b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 -- 172.21.15.181:0/439098556 shutdown_connections 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.668+0000 7f0405f40640 1 -- 172.21.15.181:0/439098556 wait complete. 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.669+0000 7f0405f40640 1 Processor -- start 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.669+0000 7f0405f40640 1 -- start start 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0400145c60 0x7f0400144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.652 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400160830 0x7f040015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f040015d0b0 con 0x7f0400071a60 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f040015cdd0 con 0x7f0400145c60 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f0405f40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04000cad70 con 0x7f0400160830 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60738/0 (socket says 172.21.15.181:60738) 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 -- 172.21.15.181:0/222752530 learned_addr learned my addr 172.21.15.181:0/222752530 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 -- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400160830 msgr2=0x7f040015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03fffff640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400160830 0x7f040015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.653 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03feffd640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0400145c60 0x7f0400144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400160830 0x7f040015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 -- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0400145c60 msgr2=0x7f0400144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0400145c60 0x7f0400144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.670+0000 7f03ff7fe640 1 -- 172.21.15.181:0/222752530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040015c120 con 0x7f0400071a60 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f03ff7fe640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b4a0 tx=0x7f03f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0014070 con 0x7f0400071a60 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f0010610 con 0x7f0400071a60 2024-03-29T04:11:35.654 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040013ddf0 con 0x7f0400071a60 2024-03-29T04:11:35.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0010a00 con 0x7f0400071a60 2024-03-29T04:11:35.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.671+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040013e3b0 con 0x7f0400071a60 2024-03-29T04:11:35.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.672+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400075a10 con 0x7f0400071a60 2024-03-29T04:11:35.655 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.672+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f03f0017070 con 0x7f0400071a60 2024-03-29T04:11:35.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.673+0000 7f03e7fff640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 0x7f03ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:35.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.673+0000 7f03feffd640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 0x7f03ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:35.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.673+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f03f008aff0 con 0x7f0400071a60 2024-03-29T04:11:35.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.673+0000 7f03feffd640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 0x7f03ec06e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0400144850 tx=0x7f03f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:35.656 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.677+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03f0059780 con 0x7f0400071a60 2024-03-29T04:11:35.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.852+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0400160460 con 0x7f0400071a60 2024-03-29T04:11:35.832 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.853+0000 7f03e7fff640 1 -- 172.21.15.181:0/222752530 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13307 (secure 0 0 0) 0x7f03f0059080 con 0x7f0400071a60 2024-03-29T04:11:35.832 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:35.833 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":44,"fsid":"fab9a006-093b-45b9-834f-2720bd7aa224","created":"2024-03-29T04:08:51.377958+0000","modified":"2024-03-29T04:11:32.451264+0000","last_up_change":"2024-03-29T04:09:02.607228+0000","last_in_change":"2024-03-29T04:08:53.637717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T04:09:03.587081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"642f3f8a-0d64-48be-9111-570fa2cfcb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6801","nonce":3549016579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6803","nonce":3549016579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6807","nonce":3549016579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":3549016579},{"type":"v1","addr":"172.21.15.181:6805","nonce":3549016579}]},"public_addr":"172.21.15.181:6801/3549016579","cluster_addr":"172.21.15.181:6803/3549016579","heartbeat_back_addr":"172.21.15.181:6807/3549016579","heartbeat_front_addr":"172.21.15.181:6805/3549016579","state":["exists","up"]},{"osd":1,"uuid":"71ca0663-03e6-45f6-b7fd-85c2eb9445ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6809","nonce":3877871096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6811","nonce":3877871096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6815","nonce":3877871096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3877871096},{"type":"v1","addr":"172.21.15.181:6813","nonce":3877871096}]},"public_addr":"172.21.15.181:6809/3877871096","cluster_addr":"172.21.15.181:6811/3877871096","heartbeat_back_addr":"172.21.15.181:6815/3877871096","heartbeat_front_addr":"172.21.15.181:6813/3877871096","state":["exists","up"]},{"osd":2,"uuid":"d06b03de-42fc-4295-a039-9514a94cb7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6817","nonce":55833287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6819","nonce":55833287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6823","nonce":55833287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":55833287},{"type":"v1","addr":"172.21.15.181:6821","nonce":55833287}]},"public_addr":"172.21.15.181:6817/55833287","cluster_addr":"172.21.15.181:6819/55833287","heartbeat_back_addr":"172.21.15.181:6823/55833287","heartbeat_front_addr":"172.21.15.181:6821/55833287","state":["exists","up"]},{"osd":3,"uuid":"659b7c6b-7766-443d-9547-d45aae9e9039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6825","nonce":3471664788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6827","nonce":3471664788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6831","nonce":3471664788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":3471664788},{"type":"v1","addr":"172.21.15.181:6829","nonce":3471664788}]},"public_addr":"172.21.15.181:6825/3471664788","cluster_addr":"172.21.15.181:6827/3471664788","heartbeat_back_addr":"172.21.15.181:6831/3471664788","heartbeat_front_addr":"172.21.15.181:6829/3471664788","state":["exists","up"]},{"osd":4,"uuid":"0a46197b-1e75-45b4-8166-96296c19c585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6801","nonce":1327517117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6803","nonce":1327517117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6807","nonce":1327517117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1327517117},{"type":"v1","addr":"172.21.15.187:6805","nonce":1327517117}]},"public_addr":"172.21.15.187:6801/1327517117","cluster_addr":"172.21.15.187:6803/1327517117","heartbeat_back_addr":"172.21.15.187:6807/1327517117","heartbeat_front_addr":"172.21.15.187:6805/1327517117","state":["exists","up"]},{"osd":5,"uuid":"ec789e34-2b4b-46fa-8f0f-37f0ddc319ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6822","nonce":461978944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6826","nonce":461978944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6831","nonce":461978944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":461978944},{"type":"v1","addr":"172.21.15.187:6829","nonce":461978944}]},"public_addr":"172.21.15.187:6822/461978944","cluster_addr":"172.21.15.187:6826/461978944","heartbeat_back_addr":"172.21.15.187:6831/461978944","heartbeat_front_addr":"172.21.15.187:6829/461978944","state":["exists","up"]},{"osd":6,"uuid":"1e1cb5e4-a484-4e65-b665-4335faa462ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6809","nonce":3602434128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6811","nonce":3602434128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6817","nonce":3602434128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3602434128},{"type":"v1","addr":"172.21.15.187:6813","nonce":3602434128}]},"public_addr":"172.21.15.187:6809/3602434128","cluster_addr":"172.21.15.187:6811/3602434128","heartbeat_back_addr":"172.21.15.187:6817/3602434128","heartbeat_front_addr":"172.21.15.187:6813/3602434128","state":["exists","up"]},{"osd":7,"uuid":"41add9a8-91cc-4862-a9a6-f9b397b77cc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6816","nonce":235935836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6819","nonce":235935836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6825","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6827","nonce":235935836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6821","nonce":235935836},{"type":"v1","addr":"172.21.15.187:6823","nonce":235935836}]},"public_addr":"172.21.15.187:6816/235935836","cluster_addr":"172.21.15.187:6819/235935836","heartbeat_back_addr":"172.21.15.187:6827/235935836","heartbeat_front_addr":"172.21.15.187:6823/235935836","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.727101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.763064+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.865365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.796857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.830325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.812175+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.864299+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T04:08:58.897425+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/2447436157":"2024-03-30T04:11:29.387620+0000","172.21.15.181:6835/2447436157":"2024-03-30T04:11:29.387620+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 msgr2=0x7f03ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 0x7f03ec06e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0400144850 tx=0x7f03f4009040 comp rx=0 tx=0).stop 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 msgr2=0x7f040015bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b4a0 tx=0x7f03f000b960 comp rx=0 tx=0).stop 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 shutdown_connections 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f03ec06c530 0x7f03ec06e9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.835 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0400160830 0x7f040015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0400145c60 0x7f0400144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 --2- 172.21.15.181:0/222752530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0400071a60 0x7f040015bbe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 >> 172.21.15.181:0/222752530 conn(0x7f04000b1c90 msgr2=0x7f04000cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 shutdown_connections 2024-03-29T04:11:35.836 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:35.855+0000 7f0405f40640 1 -- 172.21.15.181:0/222752530 wait complete. 2024-03-29T04:11:36.847 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T04:11:36.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:36.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.984+0000 7f54a6214640 1 Processor -- start 2024-03-29T04:11:36.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.984+0000 7f54a6214640 1 -- start start 2024-03-29T04:11:36.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f54a6214640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54a015e440 0x7f54a00b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f54a6214640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f54a00cf040 0x7f54a00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f54a6214640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a00d40c0 con 0x7f54a006d4a0 2024-03-29T04:11:36.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f54a6214640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f54a00d3520 con 0x7f54a015e440 2024-03-29T04:11:36.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f54a6214640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f54a00d2910 con 0x7f54a00cf040 2024-03-29T04:11:36.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54a015e440 0x7f54a00b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549effd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f54a006d4a0 0x7f54a00d39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:38180/0 (socket says 172.21.15.181:38180) 2024-03-29T04:11:36.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549effd640 1 -- 172.21.15.181:0/4148391369 learned_addr learned my addr 172.21.15.181:0/4148391369 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:36.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f54a015e440 0x7f54a00b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60754/0 (socket says 172.21.15.181:60754) 2024-03-29T04:11:36.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549ffff640 1 --2- 172.21.15.181:0/4148391369 >> v2:172.21.15.187:3300/0 conn(0x7f54a00cf040 0x7f54a00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 -- 172.21.15.181:0/4148391369 >> v2:172.21.15.187:3300/0 conn(0x7f54a00cf040 msgr2=0x7f54a00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:36.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 --2- 172.21.15.181:0/4148391369 >> v2:172.21.15.187:3300/0 conn(0x7f54a00cf040 0x7f54a00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 -- 172.21.15.181:0/4148391369 >> v1:172.21.15.181:6789/0 conn(0x7f54a006d4a0 legacy=0x7f54a00d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549f7fe640 1 -- 172.21.15.181:0/4148391369 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a00d2bf0 con 0x7f54a015e440 2024-03-29T04:11:36.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.985+0000 7f549e7fc640 1 -- 172.21.15.181:0/4148391369 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291578085 0 0) 0x7f54a00d40c0 con 0x7f54a006d4a0 2024-03-29T04:11:36.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.986+0000 7f549f7fe640 1 --2- 172.21.15.181:0/4148391369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a015e440 0x7f54a00b9ce0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f54a00d3490 tx=0x7f549002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf44ada66eb93703 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.986+0000 7f549e7fc640 1 -- 172.21.15.181:0/4148391369 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490034070 con 0x7f54a015e440 2024-03-29T04:11:36.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.986+0000 7f549e7fc640 1 -- 172.21.15.181:0/4148391369 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5490030610 con 0x7f54a015e440 2024-03-29T04:11:36.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.986+0000 7f549e7fc640 1 -- 172.21.15.181:0/4148391369 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900309e0 con 0x7f54a015e440 2024-03-29T04:11:36.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 -- 172.21.15.181:0/4148391369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a015e440 msgr2=0x7f54a00b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 --2- 172.21.15.181:0/4148391369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a015e440 0x7f54a00b9ce0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f54a00d3490 tx=0x7f549002b960 comp rx=0 tx=0).stop 2024-03-29T04:11:36.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 -- 172.21.15.181:0/4148391369 shutdown_connections 2024-03-29T04:11:36.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 --2- 172.21.15.181:0/4148391369 >> v2:172.21.15.187:3300/0 conn(0x7f54a00cf040 0x7f54a00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 --2- 172.21.15.181:0/4148391369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a015e440 0x7f54a00b9ce0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 -- 172.21.15.181:0/4148391369 >> 172.21.15.181:0/4148391369 conn(0x7f54a00676b0 msgr2=0x7f54a0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 -- 172.21.15.181:0/4148391369 shutdown_connections 2024-03-29T04:11:36.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.987+0000 7f54a6214640 1 -- 172.21.15.181:0/4148391369 wait complete. 2024-03-29T04:11:36.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.988+0000 7f54a6214640 1 Processor -- start 2024-03-29T04:11:36.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.988+0000 7f54a6214640 1 -- start start 2024-03-29T04:11:36.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 0x7f54a0151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a00cf040 0x7f54a0160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a015e440 0x7f54a015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54a00ba220 con 0x7f54a006d4a0 2024-03-29T04:11:36.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f54a00d31b0 con 0x7f54a00cf040 2024-03-29T04:11:36.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f54a6214640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f54a015e820 con 0x7f54a015e440 2024-03-29T04:11:36.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 0x7f54a0151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 0x7f54a0151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60756/0 (socket says 172.21.15.181:60756) 2024-03-29T04:11:36.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 -- 172.21.15.181:0/1649943861 learned_addr learned my addr 172.21.15.181:0/1649943861 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:36.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549effd640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a00cf040 0x7f54a0160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549ffff640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a015e440 0x7f54a015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 -- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a015e440 msgr2=0x7f54a015d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f54a015e440 0x7f54a015d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 -- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a00cf040 msgr2=0x7f54a0160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f54a00cf040 0x7f54a0160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.987 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a015ec50 con 0x7f54a006d4a0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.989+0000 7f549f7fe640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 0x7f54a0151120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f54900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.990+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490047070 con 0x7f54a006d4a0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.990+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5490030410 con 0x7f54a006d4a0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.990+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490034040 con 0x7f54a006d4a0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.990+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a015ef30 con 0x7f54a006d4a0 2024-03-29T04:11:36.988 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.990+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a015f4f0 con 0x7f54a006d4a0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.991+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54a0075900 con 0x7f54a006d4a0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.991+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f549004a070 con 0x7f54a006d4a0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f5487fff640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f548c06c560 0x7f548c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f54900ace90 con 0x7f54a006d4a0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f549effd640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f548c06c560 0x7f548c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f5487fff640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7f548c072a00 0x7f548c074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54900067f0 con 0x7f548c072a00 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f54900af590 con 0x7f54a006d4a0 2024-03-29T04:11:36.989 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f549ffff640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7f548c072a00 0x7f548c074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:36.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f549effd640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f548c06c560 0x7f548c06ea20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f54a00ca460 tx=0x7f5494007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.992+0000 7f549ffff640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7f548c072a00 0x7f548c074e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:36.990 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:36.993+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== osd.0 v2:172.21.15.181:6800/3549016579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54900067f0 con 0x7f548c072a00 2024-03-29T04:11:36.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 --> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54a01598d0 con 0x7f548c072a00 2024-03-29T04:11:36.991 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f5487fff640 1 -- 172.21.15.181:0/1649943861 <== osd.0 v2:172.21.15.181:6800/3549016579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54a01598d0 con 0x7f548c072a00 2024-03-29T04:11:36.991 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:36.991 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:36.992 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7f548c072a00 msgr2=0x7f548c074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6800/3549016579,v1:172.21.15.181:6801/3549016579] conn(0x7f548c072a00 0x7f548c074e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f548c06c560 msgr2=0x7f548c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f548c06c560 0x7f548c06ea20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f54a00ca460 tx=0x7f5494007d50 comp rx=0 tx=0).stop 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 msgr2=0x7f54a0151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.006+0000 7f54a6214640 1 --2- 172.21.15.181:0/1649943861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54a006d4a0 0x7f54a0151120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f54900046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:36.993 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.007+0000 7f549ffff640 1 -- 172.21.15.181:0/1649943861 reap_dead start 2024-03-29T04:11:36.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.007+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 shutdown_connections 2024-03-29T04:11:36.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.007+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 >> 172.21.15.181:0/1649943861 conn(0x7f54a00676b0 msgr2=0x7f54a015a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:36.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.007+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 shutdown_connections 2024-03-29T04:11:36.994 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.007+0000 7f54a6214640 1 -- 172.21.15.181:0/1649943861 wait complete. 2024-03-29T04:11:36.996 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.119+0000 7f96ba074640 1 Processor -- start 2024-03-29T04:11:37.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.119+0000 7f96ba074640 1 -- start start 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b415e7a0 0x7f96b40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 0x7f96b40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96b40d3ee0 0x7f96b40d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40cfc40 con 0x7f96b40d3ee0 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40d3120 con 0x7f96b40cec60 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96ba074640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40d3400 con 0x7f96b415e7a0 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96b3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96b40d3ee0 0x7f96b40d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96b3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96b40d3ee0 0x7f96b40d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60766/0 (socket says 172.21.15.181:60766) 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96b3fff640 1 -- 172.21.15.181:0/3938156645 learned_addr learned my addr 172.21.15.181:0/3938156645 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96aaffd640 1 --2- 172.21.15.181:0/3938156645 >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 0x7f96b40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.120+0000 7f96b37fe640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b415e7a0 0x7f96b40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 -- 172.21.15.181:0/3938156645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b415e7a0 msgr2=0x7f96b40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b415e7a0 0x7f96b40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 -- 172.21.15.181:0/3938156645 >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 msgr2=0x7f96b40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 --2- 172.21.15.181:0/3938156645 >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 0x7f96b40d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 -- 172.21.15.181:0/3938156645 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40d36e0 con 0x7f96b40d3ee0 2024-03-29T04:11:37.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96aaffd640 1 --2- 172.21.15.181:0/3938156645 >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 0x7f96b40d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b3fff640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40d3ee0 0x7f96b40d17f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f96b40d5490 tx=0x7f96ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ba69fd44201216 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.121+0000 7f96b2ffd640 1 -- 172.21.15.181:0/3938156645 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02c9e0 con 0x7f96b40d3ee0 2024-03-29T04:11:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96b2ffd640 1 -- 172.21.15.181:0/3938156645 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac02cb80 con 0x7f96b40d3ee0 2024-03-29T04:11:37.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96b2ffd640 1 -- 172.21.15.181:0/3938156645 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0343f0 con 0x7f96b40d3ee0 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 -- 172.21.15.181:0/3938156645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40d3ee0 msgr2=0x7f96b40d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40d3ee0 0x7f96b40d17f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f96b40d5490 tx=0x7f96ac02be40 comp rx=0 tx=0).stop 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 -- 172.21.15.181:0/3938156645 shutdown_connections 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40d3ee0 0x7f96b40d17f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.106 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 --2- 172.21.15.181:0/3938156645 >> v2:172.21.15.187:3300/0 conn(0x7f96b40cec60 0x7f96b40d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 --2- 172.21.15.181:0/3938156645 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b415e7a0 0x7f96b40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 -- 172.21.15.181:0/3938156645 >> 172.21.15.181:0/3938156645 conn(0x7f96b40676b0 msgr2=0x7f96b4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 -- 172.21.15.181:0/3938156645 shutdown_connections 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.122+0000 7f96ba074640 1 -- 172.21.15.181:0/3938156645 wait complete. 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.123+0000 7f96ba074640 1 Processor -- start 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.123+0000 7f96ba074640 1 -- start start 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96b40d3ee0 0x7f96b4161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.107 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b4161880 0x7f96b40cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40b5e10 con 0x7f96b40cec60 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40cf930 con 0x7f96b40d3ee0 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96ba074640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40b5f90 con 0x7f96b4161880 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60768/0 (socket says 172.21.15.181:60768) 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 -- 172.21.15.181:0/2766125475 learned_addr learned my addr 172.21.15.181:0/2766125475 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96aaffd640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96b40d3ee0 0x7f96b4161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.108 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b3fff640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b4161880 0x7f96b40cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 -- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b4161880 msgr2=0x7f96b40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b4161880 0x7f96b40cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 -- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96b40d3ee0 msgr2=0x7f96b4161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96b40d3ee0 0x7f96b4161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.124+0000 7f96b37fe640 1 -- 172.21.15.181:0/2766125475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4158070 con 0x7f96b40cec60 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96b37fe640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d620 tx=0x7f96a400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a400b620 con 0x7f96b40cec60 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4019070 con 0x7f96b40cec60 2024-03-29T04:11:37.109 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b41602a0 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a400ba00 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.125+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b4160860 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.126+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b4075900 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.126+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f96a400bba0 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.126+0000 7f96b0ff9640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 0x7f969406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.127+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f96a408b7f0 con 0x7f96b40cec60 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.127+0000 7f96aaffd640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 0x7f969406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.110 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.127+0000 7f96aaffd640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 0x7f969406e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f96b40ce580 tx=0x7f969c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.111 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.131+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f96a4059930 con 0x7f96b40cec60 2024-03-29T04:11:37.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.305+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f96b415dc50 con 0x7f969406c530 2024-03-29T04:11:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.305+0000 7f96b0ff9640 1 -- 172.21.15.181:0/2766125475 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f96b415dc50 con 0x7f969406c530 2024-03-29T04:11:37.285 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 msgr2=0x7f969406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 0x7f969406e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f96b40ce580 tx=0x7f969c008040 comp rx=0 tx=0).stop 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 msgr2=0x7f96b41f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d620 tx=0x7f96a400dae0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 shutdown_connections 2024-03-29T04:11:37.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f969406c530 0x7f969406e9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96b4161880 0x7f96b40cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96b40d3ee0 0x7f96b4161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 --2- 172.21.15.181:0/2766125475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96b40cec60 0x7f96b41f7380 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 >> 172.21.15.181:0/2766125475 conn(0x7f96b40676b0 msgr2=0x7f96b40b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 shutdown_connections 2024-03-29T04:11:37.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.307+0000 7f96ba074640 1 -- 172.21.15.181:0/2766125475 wait complete. 2024-03-29T04:11:37.297 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T04:11:37.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:37.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.423+0000 7f00fd637640 1 Processor -- start 2024-03-29T04:11:37.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.423+0000 7f00fd637640 1 -- start start 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00fd637640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00f815e440 0x7f00f80b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00fd637640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00f806d4a0 0x7f00f80d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00fd637640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f80d2910 con 0x7f00f80d3ef0 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00fd637640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f00f80d3520 con 0x7f00f806d4a0 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00fd637640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00f80d2bf0 con 0x7f00f815e440 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00f67fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00f806d4a0 0x7f00f80d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00f67fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00f806d4a0 0x7f00f80d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60780/0 (socket says 172.21.15.181:60780) 2024-03-29T04:11:37.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00f67fc640 1 -- 172.21.15.181:0/898634278 learned_addr learned my addr 172.21.15.181:0/898634278 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.424+0000 7f00f6ffd640 1 --2- 172.21.15.181:0/898634278 >> v2:172.21.15.187:3300/0 conn(0x7f00f815e440 0x7f00f80b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f67fc640 1 -- 172.21.15.181:0/898634278 >> v2:172.21.15.187:3300/0 conn(0x7f00f815e440 msgr2=0x7f00f80b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f67fc640 1 --2- 172.21.15.181:0/898634278 >> v2:172.21.15.187:3300/0 conn(0x7f00f815e440 0x7f00f80b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f67fc640 1 -- 172.21.15.181:0/898634278 >> v1:172.21.15.187:6789/0 conn(0x7f00f80d3ef0 legacy=0x7f00f80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f5ffb640 1 -- 172.21.15.181:0/898634278 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693571317 0 0) 0x7f00f80d2910 con 0x7f00f80d3ef0 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f67fc640 1 -- 172.21.15.181:0/898634278 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f80d2ed0 con 0x7f00f806d4a0 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.425+0000 7f00f67fc640 1 --2- 172.21.15.181:0/898634278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f806d4a0 0x7f00f80d39b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f00f806d430 tx=0x7f00ec02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382b22294ef3d969 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00f5ffb640 1 -- 172.21.15.181:0/898634278 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec02f850 con 0x7f00f806d4a0 2024-03-29T04:11:37.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00f5ffb640 1 -- 172.21.15.181:0/898634278 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec02f9f0 con 0x7f00f806d4a0 2024-03-29T04:11:37.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00f5ffb640 1 -- 172.21.15.181:0/898634278 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec02fdc0 con 0x7f00f806d4a0 2024-03-29T04:11:37.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 -- 172.21.15.181:0/898634278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f806d4a0 msgr2=0x7f00f80d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 --2- 172.21.15.181:0/898634278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f806d4a0 0x7f00f80d39b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f00f806d430 tx=0x7f00ec02ba30 comp rx=0 tx=0).stop 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 -- 172.21.15.181:0/898634278 shutdown_connections 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 --2- 172.21.15.181:0/898634278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f806d4a0 0x7f00f80d39b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 --2- 172.21.15.181:0/898634278 >> v2:172.21.15.187:3300/0 conn(0x7f00f815e440 0x7f00f80b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 -- 172.21.15.181:0/898634278 >> 172.21.15.181:0/898634278 conn(0x7f00f80676b0 msgr2=0x7f00f8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 -- 172.21.15.181:0/898634278 shutdown_connections 2024-03-29T04:11:37.410 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.426+0000 7f00fd637640 1 -- 172.21.15.181:0/898634278 wait complete. 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.427+0000 7f00fd637640 1 Processor -- start 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 -- start start 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f806d4a0 0x7f00f81f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 0x7f00f81540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f80c83c0 0x7f00f815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00f815e820 con 0x7f00f815e440 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80ba220 con 0x7f00f80c83c0 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00fd637640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00f815e9a0 con 0x7f00f806d4a0 2024-03-29T04:11:37.411 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00f67fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 0x7f00f81540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00f67fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 0x7f00f81540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:60786/0 (socket says 172.21.15.181:60786) 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00f67fc640 1 -- 172.21.15.181:0/3452375981 learned_addr learned my addr 172.21.15.181:0/3452375981 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00f6ffd640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f806d4a0 0x7f00f81f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.428+0000 7f00f77fe640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f80c83c0 0x7f00f815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 -- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f806d4a0 msgr2=0x7f00f81f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f806d4a0 0x7f00f81f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 -- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f80c83c0 msgr2=0x7f00f815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f80c83c0 0x7f00f815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.412 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f81f7860 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00f67fc640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 0x7f00f81540e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f00ec02ba70 tx=0x7f00ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec047070 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec037bc0 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec004030 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f80c88c0 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.429+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f80c8e80 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.430+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f00f8075900 con 0x7f00f815e440 2024-03-29T04:11:37.413 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.430+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f00ec04a070 con 0x7f00f815e440 2024-03-29T04:11:37.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00df7fe640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f00e406c530 0x7f00e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f00ec0ad570 con 0x7f00f815e440 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00f6ffd640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f00e406c530 0x7f00e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00df7fe640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f00e40729d0 0x7f00e4074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00ec038e50 con 0x7f00e40729d0 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f00ec0b4050 con 0x7f00f815e440 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.431+0000 7f00f77fe640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f00e40729d0 0x7f00e4074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.432+0000 7f00f6ffd640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f00e406c530 0x7f00e406e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00f80c7f30 tx=0x7f00e800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.415 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.432+0000 7f00f77fe640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f00e40729d0 0x7f00e4074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.432+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== osd.1 v2:172.21.15.181:6808/3877871096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00ec038e50 con 0x7f00e40729d0 2024-03-29T04:11:37.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.445+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 --> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f00f80cb150 con 0x7f00e40729d0 2024-03-29T04:11:37.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00df7fe640 1 -- 172.21.15.181:0/3452375981 <== osd.1 v2:172.21.15.181:6808/3877871096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f00f80cb150 con 0x7f00e40729d0 2024-03-29T04:11:37.425 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:37.425 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:37.426 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:37.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f00e40729d0 msgr2=0x7f00e4074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6808/3877871096,v1:172.21.15.181:6809/3877871096] conn(0x7f00e40729d0 0x7f00e4074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f00e406c530 msgr2=0x7f00e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f00e406c530 0x7f00e406e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00f80c7f30 tx=0x7f00e800b040 comp rx=0 tx=0).stop 2024-03-29T04:11:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 msgr2=0x7f00f81540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.427 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 --2- 172.21.15.181:0/3452375981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00f815e440 0x7f00f81540e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f00ec02ba70 tx=0x7f00ec004730 comp rx=0 tx=0).stop 2024-03-29T04:11:37.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00f77fe640 1 -- 172.21.15.181:0/3452375981 reap_dead start 2024-03-29T04:11:37.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 shutdown_connections 2024-03-29T04:11:37.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 >> 172.21.15.181:0/3452375981 conn(0x7f00f80676b0 msgr2=0x7f00f80b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 shutdown_connections 2024-03-29T04:11:37.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.446+0000 7f00fd637640 1 -- 172.21.15.181:0/3452375981 wait complete. 2024-03-29T04:11:37.436 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T04:11:37.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.558+0000 7f2c95041640 1 Processor -- start 2024-03-29T04:11:37.538 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.558+0000 7f2c95041640 1 -- start start 2024-03-29T04:11:37.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c95041640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 0x7f2c900cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c95041640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c900d19c0 0x7f2c900b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c95041640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c900d3120 con 0x7f2c900cec60 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c95041640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f2c900cfc40 con 0x7f2c900d19c0 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c95041640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2c900d3400 con 0x7f2c9015e7a0 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c8f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c900d19c0 0x7f2c900b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c8e59c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2c900cec60 0x7f2c900d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53958/0 (socket says 172.21.15.181:53958) 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c8e59c640 1 -- 172.21.15.181:0/3860013331 learned_addr learned my addr 172.21.15.181:0/3860013331 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.559+0000 7f2c8ed9d640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 0x7f2c900cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8f59e640 1 -- 172.21.15.181:0/3860013331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 msgr2=0x7f2c900cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8f59e640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 0x7f2c900cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8f59e640 1 -- 172.21.15.181:0/3860013331 >> v1:172.21.15.181:6789/0 conn(0x7f2c900cec60 legacy=0x7f2c900d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8f59e640 1 -- 172.21.15.181:0/3860013331 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c900d36e0 con 0x7f2c900d19c0 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8dd9b640 1 -- 172.21.15.181:0/3860013331 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805242668 0 0) 0x7f2c900d3120 con 0x7f2c900cec60 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8ed9d640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 0x7f2c900cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8f59e640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900d19c0 0x7f2c900b75a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53503eb3a5495671 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.560+0000 7f2c8dd9b640 1 -- 172.21.15.181:0/3860013331 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880315a0 con 0x7f2c900d19c0 2024-03-29T04:11:37.542 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c8dd9b640 1 -- 172.21.15.181:0/3860013331 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c88037070 con 0x7f2c900d19c0 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c8dd9b640 1 -- 172.21.15.181:0/3860013331 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88031960 con 0x7f2c900d19c0 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 -- 172.21.15.181:0/3860013331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900d19c0 msgr2=0x7f2c900b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900d19c0 0x7f2c900b75a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802b840 comp rx=0 tx=0).stop 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 -- 172.21.15.181:0/3860013331 shutdown_connections 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900d19c0 0x7f2c900b75a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 --2- 172.21.15.181:0/3860013331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c9015e7a0 0x7f2c900cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 -- 172.21.15.181:0/3860013331 >> 172.21.15.181:0/3860013331 conn(0x7f2c900676b0 msgr2=0x7f2c90067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 -- 172.21.15.181:0/3860013331 shutdown_connections 2024-03-29T04:11:37.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.561+0000 7f2c95041640 1 -- 172.21.15.181:0/3860013331 wait complete. 2024-03-29T04:11:37.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 Processor -- start 2024-03-29T04:11:37.546 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 -- start start 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c900d19c0 0x7f2c900b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c90154930 0x7f2c900b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c900d3ee0 con 0x7f2c900cec60 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c900cf930 con 0x7f2c900d19c0 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c95041640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2c900d4060 con 0x7f2c90154930 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.547 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c8ed9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44690/0 (socket says 172.21.15.181:44690) 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.563+0000 7f2c8ed9d640 1 -- 172.21.15.181:0/3393270208 learned_addr learned my addr 172.21.15.181:0/3393270208 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8e59c640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c900d19c0 0x7f2c900b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8f59e640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c90154930 0x7f2c900b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 -- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c90154930 msgr2=0x7f2c900b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c90154930 0x7f2c900b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 -- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c900d19c0 msgr2=0x7f2c900b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c900d19c0 0x7f2c900b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 -- 172.21.15.181:0/3393270208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c900b4190 con 0x7f2c900cec60 2024-03-29T04:11:37.548 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c8ed9d640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000efc0 tx=0x7f2c8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80018020 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80014680 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c900b4470 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80014a50 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.564+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c900b4a30 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.565+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c90075900 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.565+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2c80005d10 con 0x7f2c900cec60 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.566+0000 7f2c777fe640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 0x7f2c7c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.549 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.566+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f2c8008c240 con 0x7f2c900cec60 2024-03-29T04:11:37.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.566+0000 7f2c8e59c640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 0x7f2c7c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.566+0000 7f2c8e59c640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 0x7f2c7c06ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c900b9350 tx=0x7f2c84007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.550 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.570+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2c80022030 con 0x7f2c900cec60 2024-03-29T04:11:37.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.744+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2c900b6c70 con 0x7f2c7c06c560 2024-03-29T04:11:37.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.745+0000 7f2c777fe640 1 -- 172.21.15.181:0/3393270208 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2c900b6c70 con 0x7f2c7c06c560 2024-03-29T04:11:37.724 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T04:11:37.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.746+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 msgr2=0x7f2c7c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.746+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 0x7f2c7c06ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c900b9350 tx=0x7f2c84007de0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.746+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 msgr2=0x7f2c900b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.746+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000efc0 tx=0x7f2c8000c490 comp rx=0 tx=0).stop 2024-03-29T04:11:37.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 shutdown_connections 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f2c7c06c560 0x7f2c7c06ea20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2c90154930 0x7f2c900b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c900d19c0 0x7f2c900b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 --2- 172.21.15.181:0/3393270208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c900cec60 0x7f2c900b3a90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 >> 172.21.15.181:0/3393270208 conn(0x7f2c900676b0 msgr2=0x7f2c900c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 shutdown_connections 2024-03-29T04:11:37.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.747+0000 7f2c95041640 1 -- 172.21.15.181:0/3393270208 wait complete. 2024-03-29T04:11:37.737 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T04:11:37.737 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:37.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 Processor -- start 2024-03-29T04:11:37.841 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 -- start start 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55580afdf0 0x7f5558161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55580b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5558162650 con 0x7f55580b3d20 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f5558162ef0 con 0x7f55580afdf0 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.862+0000 7f555d03c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f555815f8c0 con 0x7f555815f450 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5556d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55580afdf0 0x7f5558161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.843 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f555759e640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f55580b3d20 0x7f5558154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53962/0 (socket says 172.21.15.181:53962) 2024-03-29T04:11:37.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f555759e640 1 -- 172.21.15.181:0/3971300352 learned_addr learned my addr 172.21.15.181:0/3971300352 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:37.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f555659c640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55580b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5556d9d640 1 -- 172.21.15.181:0/3971300352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 msgr2=0x7f55580b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5556d9d640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55580b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.844 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5556d9d640 1 -- 172.21.15.181:0/3971300352 >> v1:172.21.15.181:6789/0 conn(0x7f55580b3d20 legacy=0x7f5558154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5556d9d640 1 -- 172.21.15.181:0/3971300352 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558162930 con 0x7f55580afdf0 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.863+0000 7f5555d9b640 1 -- 172.21.15.181:0/3971300352 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982008887 0 0) 0x7f5558162650 con 0x7f55580b3d20 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f5556d9d640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580afdf0 0x7f5558161fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55480097a0 tx=0x7f554802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2513f8eff06469e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f5555d9b640 1 -- 172.21.15.181:0/3971300352 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548035020 con 0x7f55580afdf0 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f5555d9b640 1 -- 172.21.15.181:0/3971300352 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548031830 con 0x7f55580afdf0 2024-03-29T04:11:37.845 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f5555d9b640 1 -- 172.21.15.181:0/3971300352 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548031c00 con 0x7f55580afdf0 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 -- 172.21.15.181:0/3971300352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580afdf0 msgr2=0x7f5558161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580afdf0 0x7f5558161fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55480097a0 tx=0x7f554802b8d0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 -- 172.21.15.181:0/3971300352 shutdown_connections 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55580b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 --2- 172.21.15.181:0/3971300352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580afdf0 0x7f5558161fc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 -- 172.21.15.181:0/3971300352 >> 172.21.15.181:0/3971300352 conn(0x7f55580d0870 msgr2=0x7f55580d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.849 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 -- 172.21.15.181:0/3971300352 shutdown_connections 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.864+0000 7f555d03c640 1 -- 172.21.15.181:0/3971300352 wait complete. 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 Processor -- start 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 -- start start 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55580afdf0 0x7f555815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 0x7f5558159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55581610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f555806b720 con 0x7f55580b3d20 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.866+0000 7f555d03c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55580b4150 con 0x7f55580afdf0 2024-03-29T04:11:37.850 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555d03c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55580b01d0 con 0x7f555815f450 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 0x7f5558159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 0x7f5558159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44706/0 (socket says 172.21.15.181:44706) 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 -- 172.21.15.181:0/1968219690 learned_addr learned my addr 172.21.15.181:0/1968219690 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f5556d9d640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55580afdf0 0x7f555815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555759e640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55581610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 -- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 msgr2=0x7f55581610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f555815f450 0x7f55581610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 -- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55580afdf0 msgr2=0x7f555815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55580afdf0 0x7f555815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55580cdf90 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555659c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 0x7f5558159000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f554000d620 tx=0x7f554000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000b620 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5540019070 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555815bf30 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000ba00 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.867+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55580ce530 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.868+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5558075900 con 0x7f55580b3d20 2024-03-29T04:11:37.852 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f554000bba0 con 0x7f55580b3d20 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f553806c530 0x7f553806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f554008b8e0 con 0x7f55580b3d20 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f5556d9d640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f553806c530 0x7f553806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7f55380729d0 0x7f5538074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55380754e0 con 0x7f55380729d0 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.869+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f55400902c0 con 0x7f55580b3d20 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.870+0000 7f555759e640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7f55380729d0 0x7f5538074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.853 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.870+0000 7f5556d9d640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f553806c530 0x7f553806e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5548036000 tx=0x7f554803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.870+0000 7f555759e640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7f55380729d0 0x7f5538074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.870+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== osd.2 v2:172.21.15.181:6816/55833287 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55380754e0 con 0x7f55380729d0 2024-03-29T04:11:37.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.883+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 --> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5558067690 con 0x7f55380729d0 2024-03-29T04:11:37.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.883+0000 7f554f7fe640 1 -- 172.21.15.181:0/1968219690 <== osd.2 v2:172.21.15.181:6816/55833287 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5558067690 con 0x7f55380729d0 2024-03-29T04:11:37.863 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:37.863 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:37.863 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7f55380729d0 msgr2=0x7f5538074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6816/55833287,v1:172.21.15.181:6817/55833287] conn(0x7f55380729d0 0x7f5538074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f553806c530 msgr2=0x7f553806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f553806c530 0x7f553806e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5548036000 tx=0x7f554803d040 comp rx=0 tx=0).stop 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 msgr2=0x7f5558159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 --2- 172.21.15.181:0/1968219690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55580b3d20 0x7f5558159000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f554000d620 tx=0x7f554000dae0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555759e640 1 -- 172.21.15.181:0/1968219690 reap_dead start 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 shutdown_connections 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 >> 172.21.15.181:0/1968219690 conn(0x7f55580d0870 msgr2=0x7f555806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 shutdown_connections 2024-03-29T04:11:37.865 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.884+0000 7f555d03c640 1 -- 172.21.15.181:0/1968219690 wait complete. 2024-03-29T04:11:37.873 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T04:11:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 Processor -- start 2024-03-29T04:11:37.975 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 -- start start 2024-03-29T04:11:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbe015e7a0 0x7fcbe00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbe00cec60 0x7fcbe00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe00d3120 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.976 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe00cfc40 con 0x7fcbe00cec60 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbe4d29640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe00d3400 con 0x7fcbe015e7a0 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbd7fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbe00cec60 0x7fcbe00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbd7fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbe00cec60 0x7fcbe00d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44710/0 (socket says 172.21.15.181:44710) 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.996+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2566186462 learned_addr learned my addr 172.21.15.181:0/2566186462 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.997+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2566186462 >> v2:172.21.15.187:3300/0 conn(0x7fcbe015e7a0 msgr2=0x7fcbe00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.997+0000 7fcbd7fff640 1 --2- 172.21.15.181:0/2566186462 >> v2:172.21.15.187:3300/0 conn(0x7fcbe015e7a0 0x7fcbe00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.997+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2566186462 >> v1:172.21.15.187:6789/0 conn(0x7fcbe00d3ee0 legacy=0x7fcbe00b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:37.977 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.997+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2566186462 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe00d36e0 con 0x7fcbe00cec60 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.997+0000 7fcbd7fff640 1 --2- 172.21.15.181:0/2566186462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00cec60 0x7fcbe00d12b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcbe006d8f0 tx=0x7fcbc802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a8f760cd6e7ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbddd9b640 1 -- 172.21.15.181:0/2566186462 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc802e070 con 0x7fcbe00cec60 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbddd9b640 1 -- 172.21.15.181:0/2566186462 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc8037070 con 0x7fcbe00cec60 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbddd9b640 1 -- 172.21.15.181:0/2566186462 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8031a00 con 0x7fcbe00cec60 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2566186462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00cec60 msgr2=0x7fcbe00d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2566186462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00cec60 0x7fcbe00d12b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcbe006d8f0 tx=0x7fcbc802b880 comp rx=0 tx=0).stop 2024-03-29T04:11:37.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2566186462 shutdown_connections 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2566186462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00cec60 0x7fcbe00d12b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2566186462 >> v2:172.21.15.187:3300/0 conn(0x7fcbe015e7a0 0x7fcbe00cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2566186462 >> 172.21.15.181:0/2566186462 conn(0x7fcbe00676b0 msgr2=0x7fcbe0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2566186462 shutdown_connections 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:37.998+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2566186462 wait complete. 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 Processor -- start 2024-03-29T04:11:37.980 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 -- start start 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe00cec60 0x7fcbe01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe00b4f40 0x7fcbe015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe00d17f0 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe00cf930 con 0x7fcbe00b4f40 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbe4d29640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe00d1970 con 0x7fcbe00cec60 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbd7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44718/0 (socket says 172.21.15.181:44718) 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.000+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2978988917 learned_addr learned my addr 172.21.15.181:0/2978988917 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbde59c640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe00cec60 0x7fcbe01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbded9d640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe00b4f40 0x7fcbe015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe00cec60 msgr2=0x7fcbe01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe00cec60 0x7fcbe01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe00b4f40 msgr2=0x7fcbe015a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe00b4f40 0x7fcbe015a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 -- 172.21.15.181:0/2978988917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe015a980 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.984 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd7fff640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000c00 tx=0x7fcbc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8032880 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc8032a20 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8032e10 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe00b5700 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.001+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe00b4050 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.002+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe00cad20 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.002+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcbc80453f0 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.003+0000 7fcbd6ffd640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 0x7fcbc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:37.985 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.003+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fcbc80ad490 con 0x7fcbe00d3ee0 2024-03-29T04:11:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.003+0000 7fcbde59c640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 0x7fcbc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.003+0000 7fcbde59c640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 0x7fcbc006e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098e0 tx=0x7fcbd0009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:37.986 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.007+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcbc8040070 con 0x7fcbe00d3ee0 2024-03-29T04:11:38.161 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.182+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fcbe0075900 con 0x7fcbc006c530 2024-03-29T04:11:38.162 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.182+0000 7fcbd6ffd640 1 -- 172.21.15.181:0/2978988917 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcbe0075900 con 0x7fcbc006c530 2024-03-29T04:11:38.162 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T04:11:38.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 msgr2=0x7fcbc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 0x7fcbc006e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098e0 tx=0x7fcbd0009060 comp rx=0 tx=0).stop 2024-03-29T04:11:38.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 msgr2=0x7fcbe00b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000c00 tx=0x7fcbc80046c0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.164 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 shutdown_connections 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe00b4f40 0x7fcbe015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbe00d3ee0 0x7fcbe00b4850 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fcbc006c530 0x7fcbc006e9f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 --2- 172.21.15.181:0/2978988917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe00cec60 0x7fcbe01609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 >> 172.21.15.181:0/2978988917 conn(0x7fcbe00676b0 msgr2=0x7fcbe00c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.184+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 shutdown_connections 2024-03-29T04:11:38.165 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.185+0000 7fcbe4d29640 1 -- 172.21.15.181:0/2978988917 wait complete. 2024-03-29T04:11:38.175 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T04:11:38.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:38.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.300+0000 7f714111c640 1 Processor -- start 2024-03-29T04:11:38.280 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.300+0000 7f714111c640 1 -- start start 2024-03-29T04:11:38.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f714111c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f713c145770 0x7f713c160a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f714111c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f713c06bc90 0x7f713c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f714111c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713c145b50 con 0x7f713c06f980 2024-03-29T04:11:38.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f714111c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f713c146070 con 0x7f713c145770 2024-03-29T04:11:38.281 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f714111c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f713c160f70 con 0x7f713c06bc90 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713ad9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f713c145770 0x7f713c160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713a59c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f713c06f980 0x7f713c06b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53966/0 (socket says 172.21.15.181:53966) 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713a59c640 1 -- 172.21.15.181:0/1431604071 learned_addr learned my addr 172.21.15.181:0/1431604071 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713b59e640 1 --2- 172.21.15.181:0/1431604071 >> v2:172.21.15.187:3300/0 conn(0x7f713c06bc90 0x7f713c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713ad9d640 1 -- 172.21.15.181:0/1431604071 >> v2:172.21.15.187:3300/0 conn(0x7f713c06bc90 msgr2=0x7f713c0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713ad9d640 1 --2- 172.21.15.181:0/1431604071 >> v2:172.21.15.187:3300/0 conn(0x7f713c06bc90 0x7f713c0728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713ad9d640 1 -- 172.21.15.181:0/1431604071 >> v1:172.21.15.181:6789/0 conn(0x7f713c06f980 legacy=0x7f713c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f7139d9b640 1 -- 172.21.15.181:0/1431604071 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695028582 0 0) 0x7f713c145b50 con 0x7f713c06f980 2024-03-29T04:11:38.283 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.301+0000 7f713ad9d640 1 -- 172.21.15.181:0/1431604071 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c0d0370 con 0x7f713c145770 2024-03-29T04:11:38.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.302+0000 7f713ad9d640 1 --2- 172.21.15.181:0/1431604071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c145770 0x7f713c160a30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f713c0b6460 tx=0x7f712c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c65ff0c6728026 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.302+0000 7f7139d9b640 1 -- 172.21.15.181:0/1431604071 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c039070 con 0x7f713c145770 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.302+0000 7f7139d9b640 1 -- 172.21.15.181:0/1431604071 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c02cbf0 con 0x7f713c145770 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.302+0000 7f7139d9b640 1 -- 172.21.15.181:0/1431604071 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c034440 con 0x7f713c145770 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 -- 172.21.15.181:0/1431604071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c145770 msgr2=0x7f713c160a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 --2- 172.21.15.181:0/1431604071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c145770 0x7f713c160a30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f713c0b6460 tx=0x7f712c02be40 comp rx=0 tx=0).stop 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 -- 172.21.15.181:0/1431604071 shutdown_connections 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 --2- 172.21.15.181:0/1431604071 >> v2:172.21.15.187:3300/0 conn(0x7f713c06bc90 0x7f713c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 --2- 172.21.15.181:0/1431604071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c145770 0x7f713c160a30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 -- 172.21.15.181:0/1431604071 >> 172.21.15.181:0/1431604071 conn(0x7f713c0d10e0 msgr2=0x7f713c0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 -- 172.21.15.181:0/1431604071 shutdown_connections 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.303+0000 7f714111c640 1 -- 172.21.15.181:0/1431604071 wait complete. 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.304+0000 7f714111c640 1 Processor -- start 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.304+0000 7f714111c640 1 -- start start 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 0x7f713c141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f713c06f980 0x7f713c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f713c145770 0x7f713c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f713c0d0c10 con 0x7f713c06bc90 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f713c0d0930 con 0x7f713c145770 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f714111c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f713c0cfdb0 con 0x7f713c06f980 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 0x7f713c141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 0x7f713c141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44736/0 (socket says 172.21.15.181:44736) 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 -- 172.21.15.181:0/3177655054 learned_addr learned my addr 172.21.15.181:0/3177655054 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713a59c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f713c06f980 0x7f713c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713b59e640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f713c145770 0x7f713c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 -- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f713c06f980 msgr2=0x7f713c15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f713c06f980 0x7f713c15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 -- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f713c145770 msgr2=0x7f713c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f713c145770 0x7f713c15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.305+0000 7f713ad9d640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c158800 con 0x7f713c06bc90 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f713ad9d640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 0x7f713c141ca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f712c009e80 tx=0x7f712c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c039050 con 0x7f713c06bc90 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c045070 con 0x7f713c06bc90 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c03e850 con 0x7f713c06bc90 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c158ae0 con 0x7f713c06bc90 2024-03-29T04:11:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.306+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c1421e0 con 0x7f713c06bc90 2024-03-29T04:11:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.307+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f713c15ce60 con 0x7f713c06bc90 2024-03-29T04:11:38.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.307+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f712c03e9f0 con 0x7f713c06bc90 2024-03-29T04:11:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f71237fe640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f712806c560 0x7f712806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f712c0ae240 con 0x7f713c06bc90 2024-03-29T04:11:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f713a59c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f712806c560 0x7f712806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f71237fe640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f7128072a00 0x7f7128074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7128075510 con 0x7f7128072a00 2024-03-29T04:11:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f712c0b5050 con 0x7f713c06bc90 2024-03-29T04:11:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f713b59e640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f7128072a00 0x7f7128074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f713a59c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f712806c560 0x7f712806ea20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7130007f70 tx=0x7f7130007f00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.308+0000 7f713b59e640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f7128072a00 0x7f7128074e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.309+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== osd.3 v2:172.21.15.181:6824/3471664788 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7128075510 con 0x7f7128072a00 2024-03-29T04:11:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 --> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f713c156110 con 0x7f7128072a00 2024-03-29T04:11:38.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f71237fe640 1 -- 172.21.15.181:0/3177655054 <== osd.3 v2:172.21.15.181:6824/3471664788 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f713c156110 con 0x7f7128072a00 2024-03-29T04:11:38.301 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:38.301 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:38.302 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f7128072a00 msgr2=0x7f7128074e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6824/3471664788,v1:172.21.15.181:6825/3471664788] conn(0x7f7128072a00 0x7f7128074e40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f712806c560 msgr2=0x7f712806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f712806c560 0x7f712806ea20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7130007f70 tx=0x7f7130007f00 comp rx=0 tx=0).stop 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 msgr2=0x7f713c141ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 --2- 172.21.15.181:0/3177655054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f713c06bc90 0x7f713c141ca0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f712c009e80 tx=0x7f712c004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f713b59e640 1 -- 172.21.15.181:0/3177655054 reap_dead start 2024-03-29T04:11:38.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 shutdown_connections 2024-03-29T04:11:38.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 >> 172.21.15.181:0/3177655054 conn(0x7f713c0d10e0 msgr2=0x7f713c0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 shutdown_connections 2024-03-29T04:11:38.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.322+0000 7f714111c640 1 -- 172.21.15.181:0/3177655054 wait complete. 2024-03-29T04:11:38.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T04:11:38.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 Processor -- start 2024-03-29T04:11:38.414 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 -- start start 2024-03-29T04:11:38.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204415e7a0 0x7f20440cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.416 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20440d19c0 0x7f20440b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20440d3120 con 0x7f20440cec60 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f20440cfc40 con 0x7f204415e7a0 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f20496b3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20440d3400 con 0x7f20440d19c0 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f2042ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204415e7a0 0x7f20440cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.435+0000 7f2042ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f204415e7a0 0x7f20440cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44746/0 (socket says 172.21.15.181:44746) 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 -- 172.21.15.181:0/4014909011 learned_addr learned my addr 172.21.15.181:0/4014909011 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f20437fe640 1 --2- 172.21.15.181:0/4014909011 >> v2:172.21.15.187:3300/0 conn(0x7f20440d19c0 0x7f20440b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 -- 172.21.15.181:0/4014909011 >> v2:172.21.15.187:3300/0 conn(0x7f20440d19c0 msgr2=0x7f20440b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.417 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 --2- 172.21.15.181:0/4014909011 >> v2:172.21.15.187:3300/0 conn(0x7f20440d19c0 0x7f20440b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 -- 172.21.15.181:0/4014909011 >> v1:172.21.15.187:6789/0 conn(0x7f20440cec60 legacy=0x7f20440d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 -- 172.21.15.181:0/4014909011 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20440d36e0 con 0x7f204415e7a0 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.436+0000 7f2042ffd640 1 --2- 172.21.15.181:0/4014909011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204415e7a0 0x7f20440cf3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f20440cf320 tx=0x7f203402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee686d225a3c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f2041ffb640 1 -- 172.21.15.181:0/4014909011 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203402ca80 con 0x7f204415e7a0 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f2041ffb640 1 -- 172.21.15.181:0/4014909011 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203402cc20 con 0x7f204415e7a0 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f2041ffb640 1 -- 172.21.15.181:0/4014909011 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20340343f0 con 0x7f204415e7a0 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 -- 172.21.15.181:0/4014909011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204415e7a0 msgr2=0x7f20440cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.418 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 --2- 172.21.15.181:0/4014909011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204415e7a0 0x7f20440cf3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f20440cf320 tx=0x7f203402bee0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 -- 172.21.15.181:0/4014909011 shutdown_connections 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 --2- 172.21.15.181:0/4014909011 >> v2:172.21.15.187:3300/0 conn(0x7f20440d19c0 0x7f20440b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 --2- 172.21.15.181:0/4014909011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204415e7a0 0x7f20440cf3f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 -- 172.21.15.181:0/4014909011 >> 172.21.15.181:0/4014909011 conn(0x7f20440676b0 msgr2=0x7f2044067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 -- 172.21.15.181:0/4014909011 shutdown_connections 2024-03-29T04:11:38.419 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.437+0000 7f20496b3640 1 -- 172.21.15.181:0/4014909011 wait complete. 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 Processor -- start 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 -- start start 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20440cec60 0x7f20441f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20440d19c0 0x7f20441f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20440d3ee0 con 0x7f2044153b10 2024-03-29T04:11:38.422 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20440cf930 con 0x7f20440d19c0 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20496b3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20440d4060 con 0x7f20440cec60 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20437fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20437fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44758/0 (socket says 172.21.15.181:44758) 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.439+0000 7f20437fe640 1 -- 172.21.15.181:0/840213530 learned_addr learned my addr 172.21.15.181:0/840213530 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f2042ffd640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20440cec60 0x7f20441f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20427fc640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20440d19c0 0x7f20441f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 -- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20440cec60 msgr2=0x7f20441f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.423 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20440cec60 0x7f20441f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 -- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20440d19c0 msgr2=0x7f20441f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20440d19c0 0x7f20441f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 -- 172.21.15.181:0/840213530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20441f5000 con 0x7f2044153b10 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20437fe640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e7f0 tx=0x7f203c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c017070 con 0x7f2044153b10 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c00c730 con 0x7f2044153b10 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20441f52e0 con 0x7f2044153b10 2024-03-29T04:11:38.424 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00cb20 con 0x7f2044153b10 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.440+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2044153fa0 con 0x7f2044153b10 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.441+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2044075900 con 0x7f2044153b10 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.441+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f203c01a070 con 0x7f2044153b10 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.442+0000 7f202b7fe640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 0x7f203006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.442+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f203c08b770 con 0x7f2044153b10 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.442+0000 7f2042ffd640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 0x7f203006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.425 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.442+0000 7f2042ffd640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 0x7f203006e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2034035000 tx=0x7f2034030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.446+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f203c059f10 con 0x7f2044153b10 2024-03-29T04:11:38.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.623+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f20441f8e60 con 0x7f203006c530 2024-03-29T04:11:38.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.623+0000 7f202b7fe640 1 -- 172.21.15.181:0/840213530 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f203402c900 con 0x7f203006c530 2024-03-29T04:11:38.603 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 msgr2=0x7f203006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 0x7f203006e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2034035000 tx=0x7f2034030000 comp rx=0 tx=0).stop 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 msgr2=0x7f2044160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e7f0 tx=0x7f203c00ecb0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 shutdown_connections 2024-03-29T04:11:38.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2044153b10 0x7f2044160180 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20440d19c0 0x7f20441f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f203006c530 0x7f203006e9f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 --2- 172.21.15.181:0/840213530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20440cec60 0x7f20441f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 >> 172.21.15.181:0/840213530 conn(0x7f20440676b0 msgr2=0x7f2044067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 shutdown_connections 2024-03-29T04:11:38.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.625+0000 7f20496b3640 1 -- 172.21.15.181:0/840213530 wait complete. 2024-03-29T04:11:38.615 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T04:11:38.616 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:38.719 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.740+0000 7f3432088640 1 Processor -- start 2024-03-29T04:11:38.720 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.740+0000 7f3432088640 1 -- start start 2024-03-29T04:11:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f3432088640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f342c0cd950 0x7f342c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f3432088640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c0b2fc0 con 0x7f342c0d03e0 2024-03-29T04:11:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f3432088640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c0b32a0 con 0x7f342c144a80 2024-03-29T04:11:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f3432088640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0d0830 con 0x7f342c0cd950 2024-03-29T04:11:38.723 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f342bfff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f342c0d03e0 0x7f342c0729a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53980/0 (socket says 172.21.15.181:53980) 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f342bfff640 1 -- 172.21.15.181:0/339889469 learned_addr learned my addr 172.21.15.181:0/339889469 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184598026 0 0) 0x7f342c0b2fc0 con 0x7f342c0d03e0 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3414003520 con 0x7f342c0d03e0 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.741+0000 7f342affd640 1 --2- 172.21.15.181:0/339889469 >> v2:172.21.15.187:3300/0 conn(0x7f342c0cd950 0x7f342c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2159122103 0 0) 0x7f3414003520 con 0x7f342c0d03e0 2024-03-29T04:11:38.724 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 >> v2:172.21.15.187:3300/0 conn(0x7f342c0cd950 msgr2=0x7f342c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 --2- 172.21.15.181:0/339889469 >> v2:172.21.15.187:3300/0 conn(0x7f342c0cd950 0x7f342c0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 >> v1:172.21.15.187:6789/0 conn(0x7f342c144a80 legacy=0x7f342c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c0b3580 con 0x7f342c0d03e0 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602138671 0 0) 0x7f34240019f0 con 0x7f342c0d03e0 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 >> v1:172.21.15.181:6789/0 conn(0x7f342c0d03e0 legacy=0x7f342c0729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f3414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3414004800 0x7f3414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.725 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34140051c0 0x7f341400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0d0830 con 0x7f34140039b0 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f341400c060 con 0x7f34140051c0 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f341400c260 con 0x7f3414004800 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f3414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342bfff640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34140051c0 0x7f341400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342affd640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3414004800 0x7f3414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 -- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3414004800 msgr2=0x7f3414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3414004800 0x7f3414004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.726 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 -- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34140051c0 msgr2=0x7f341400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34140051c0 0x7f341400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342b7fe640 1 -- 172.21.15.181:0/339889469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3414003520 con 0x7f34140039b0 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.742+0000 7f342bfff640 1 -- 172.21.15.181:0/339889469 reap_dead start 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.743+0000 7f342b7fe640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f3414003dd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f341c001750 tx=0x7f3414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0affa388016c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.743+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c001370 con 0x7f34140039b0 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.743+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c0052d0 con 0x7f34140039b0 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.743+0000 7f342a7fc640 1 -- 172.21.15.181:0/339889469 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c003830 con 0x7f34140039b0 2024-03-29T04:11:38.727 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 -- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 msgr2=0x7f3414003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f3414003dd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f341c001750 tx=0x7f3414005830 comp rx=0 tx=0).stop 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 -- 172.21.15.181:0/339889469 shutdown_connections 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 --2- 172.21.15.181:0/339889469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f3414003dd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 -- 172.21.15.181:0/339889469 >> 172.21.15.181:0/339889469 conn(0x7f342c06f060 msgr2=0x7f342c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 -- 172.21.15.181:0/339889469 shutdown_connections 2024-03-29T04:11:38.728 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.744+0000 7f3432088640 1 -- 172.21.15.181:0/339889469 wait complete. 2024-03-29T04:11:38.731 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.745+0000 7f3432088640 1 Processor -- start 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.745+0000 7f3432088640 1 -- start start 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f342c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342c15c1e0 0x7f342c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342c142360 0x7f342c1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f342c144e60 con 0x7f34140039b0 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0b3b40 con 0x7f342c142360 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f342c071620 con 0x7f342c15c1e0 2024-03-29T04:11:38.732 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f342c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f342c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44772/0 (socket says 172.21.15.181:44772) 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 -- 172.21.15.181:0/479389474 learned_addr learned my addr 172.21.15.181:0/479389474 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342bfff640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342c142360 0x7f342c1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342affd640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342c15c1e0 0x7f342c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 -- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342c15c1e0 msgr2=0x7f342c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f342c15c1e0 0x7f342c15c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 -- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342c142360 msgr2=0x7f342c1427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.733 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f342c142360 0x7f342c1427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1430d0 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f342b7fe640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f342c15aa20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f341c005040 tx=0x7f341c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c041070 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c005480 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1433b0 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c007a30 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.746+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c143970 con 0x7f34140039b0 2024-03-29T04:11:38.734 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.747+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f342c075900 con 0x7f34140039b0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f341c044070 con 0x7f34140039b0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f341806c530 0x7f341806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f341c0ad070 con 0x7f34140039b0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f342affd640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f341806c530 0x7f341806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7f34180729d0 0x7f3418074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f341c0052d0 con 0x7f34180729d0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.748+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f341c0b4050 con 0x7f34140039b0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.749+0000 7f342bfff640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7f34180729d0 0x7f3418074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.749+0000 7f342affd640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f341806c530 0x7f341806e9f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f342c0b5070 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.749+0000 7f342bfff640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7f34180729d0 0x7f3418074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.736 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.750+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== osd.4 v2:172.21.15.187:6800/1327517117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f341c0052d0 con 0x7f34180729d0 2024-03-29T04:11:38.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.763+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 --> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f342c0726f0 con 0x7f34180729d0 2024-03-29T04:11:38.742 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.763+0000 7f3413fff640 1 -- 172.21.15.181:0/479389474 <== osd.4 v2:172.21.15.187:6800/1327517117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f342c0726f0 con 0x7f34180729d0 2024-03-29T04:11:38.742 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:38.743 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:38.743 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:38.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.763+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7f34180729d0 msgr2=0x7f3418074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.763+0000 7f3432088640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.187:6800/1327517117,v1:172.21.15.187:6801/1327517117] conn(0x7f34180729d0 0x7f3418074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f341806c530 msgr2=0x7f341806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f341806c530 0x7f341806e9f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f342c0b5070 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 msgr2=0x7f342c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 --2- 172.21.15.181:0/479389474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34140039b0 0x7f342c15aa20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f341c005040 tx=0x7f341c001120 comp rx=0 tx=0).stop 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f342bfff640 1 -- 172.21.15.181:0/479389474 reap_dead start 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 shutdown_connections 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 >> 172.21.15.181:0/479389474 conn(0x7f342c06f060 msgr2=0x7f342c0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 shutdown_connections 2024-03-29T04:11:38.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.764+0000 7f3432088640 1 -- 172.21.15.181:0/479389474 wait complete. 2024-03-29T04:11:38.753 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T04:11:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 Processor -- start 2024-03-29T04:11:38.854 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 -- start start 2024-03-29T04:11:38.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe64815f890 0x7fe648151c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64815feb0 con 0x7fe6480cbd00 2024-03-29T04:11:38.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe64815c480 con 0x7fe64815f890 2024-03-29T04:11:38.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.875+0000 7fe64d055640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480cc140 con 0x7fe6480cb050 2024-03-29T04:11:38.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe64759e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe64815f890 0x7fe648151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:46800/0 (socket says 172.21.15.181:46800) 2024-03-29T04:11:38.860 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 -- 172.21.15.181:0/4149427583 learned_addr learned my addr 172.21.15.181:0/4149427583 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 -- 172.21.15.181:0/4149427583 >> v2:172.21.15.187:3300/0 conn(0x7fe64815f890 msgr2=0x7fe648151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 --2- 172.21.15.181:0/4149427583 >> v2:172.21.15.187:3300/0 conn(0x7fe64815f890 0x7fe648151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 -- 172.21.15.181:0/4149427583 >> v1:172.21.15.187:6789/0 conn(0x7fe6480cbd00 legacy=0x7fe64815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.876+0000 7fe646d9d640 1 -- 172.21.15.181:0/4149427583 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64815c910 con 0x7fe6480cb050 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.877+0000 7fe646d9d640 1 --2- 172.21.15.181:0/4149427583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a7bb1235367701d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.877+0000 7fe645d9b640 1 -- 172.21.15.181:0/4149427583 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c034070 con 0x7fe6480cb050 2024-03-29T04:11:38.861 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.877+0000 7fe645d9b640 1 -- 172.21.15.181:0/4149427583 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c030840 con 0x7fe6480cb050 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.877+0000 7fe645d9b640 1 -- 172.21.15.181:0/4149427583 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c030c10 con 0x7fe6480cb050 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 -- 172.21.15.181:0/4149427583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 msgr2=0x7fe6480cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 --2- 172.21.15.181:0/4149427583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c02b980 comp rx=0 tx=0).stop 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 -- 172.21.15.181:0/4149427583 shutdown_connections 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 --2- 172.21.15.181:0/4149427583 >> v2:172.21.15.187:3300/0 conn(0x7fe64815f890 0x7fe648151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 --2- 172.21.15.181:0/4149427583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6480cb050 0x7fe6480cb7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 -- 172.21.15.181:0/4149427583 >> 172.21.15.181:0/4149427583 conn(0x7fe64806d610 msgr2=0x7fe64806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 -- 172.21.15.181:0/4149427583 shutdown_connections 2024-03-29T04:11:38.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.878+0000 7fe64d055640 1 -- 172.21.15.181:0/4149427583 wait complete. 2024-03-29T04:11:38.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.879+0000 7fe64d055640 1 Processor -- start 2024-03-29T04:11:38.863 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.879+0000 7fe64d055640 1 -- start start 2024-03-29T04:11:38.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6480cbd00 0x7fe6480c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64815f890 0x7fe6481535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.866 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe64815c110 con 0x7fe6480cb050 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe64815be30 con 0x7fe6480cbd00 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64d055640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480cb430 con 0x7fe64815f890 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44774/0 (socket says 172.21.15.181:44774) 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 -- 172.21.15.181:0/952609028 learned_addr learned my addr 172.21.15.181:0/952609028 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64759e640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64815f890 0x7fe6481535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe64659c640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6480cbd00 0x7fe6480c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.867 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 -- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64815f890 msgr2=0x7fe6481535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64815f890 0x7fe6481535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 -- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6480cbd00 msgr2=0x7fe6480c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:38.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6480cbd00 0x7fe6480c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:38.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.880+0000 7fe646d9d640 1 -- 172.21.15.181:0/952609028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481573b0 con 0x7fe6480cb050 2024-03-29T04:11:38.868 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe646d9d640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c048070 con 0x7fe6480cb050 2024-03-29T04:11:38.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c030640 con 0x7fe6480cb050 2024-03-29T04:11:38.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6480b35a0 con 0x7fe6480cb050 2024-03-29T04:11:38.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c034040 con 0x7fe6480cb050 2024-03-29T04:11:38.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.881+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6480b3b60 con 0x7fe6480cb050 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.882+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe648075900 con 0x7fe6480cb050 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.882+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe63c04b070 con 0x7fe6480cb050 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.883+0000 7fe62f7fe640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 0x7fe63406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.883+0000 7fe64659c640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 0x7fe63406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.883+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe63c0ad470 con 0x7fe6480cb050 2024-03-29T04:11:38.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.883+0000 7fe64659c640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 0x7fe63406e9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe64815ac50 tx=0x7fe638009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:38.871 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:38.887+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe63c07bc00 con 0x7fe6480cb050 2024-03-29T04:11:39.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.061+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe64806f6b0 con 0x7fe63406c530 2024-03-29T04:11:39.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.062+0000 7fe62f7fe640 1 -- 172.21.15.181:0/952609028 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe64806f6b0 con 0x7fe63406c530 2024-03-29T04:11:39.041 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T04:11:39.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 msgr2=0x7fe63406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 0x7fe63406e9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe64815ac50 tx=0x7fe638009040 comp rx=0 tx=0).stop 2024-03-29T04:11:39.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 msgr2=0x7fe6480b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe63c000c00 tx=0x7fe63c004730 comp rx=0 tx=0).stop 2024-03-29T04:11:39.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 shutdown_connections 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe63406c530 0x7fe63406e9f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64815f890 0x7fe6481535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6480cbd00 0x7fe6480c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 --2- 172.21.15.181:0/952609028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480cb050 0x7fe6480b2ce0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.063+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 >> 172.21.15.181:0/952609028 conn(0x7fe64806d610 msgr2=0x7fe6480cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.064+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 shutdown_connections 2024-03-29T04:11:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.064+0000 7fe64d055640 1 -- 172.21.15.181:0/952609028 wait complete. 2024-03-29T04:11:39.053 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T04:11:39.054 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:39.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.179+0000 7fb746fba640 1 Processor -- start 2024-03-29T04:11:39.159 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.179+0000 7fb746fba640 1 -- start start 2024-03-29T04:11:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb746fba640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 0x7fb7400d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb746fba640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7400cd280 0x7fb7400cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb746fba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7400ccf00 con 0x7fb7400cc9d0 2024-03-29T04:11:39.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb746fba640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fb7400cdc10 con 0x7fb7400cd280 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb746fba640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb7400cf6c0 con 0x7fb7400af990 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb745557640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7400cd280 0x7fb7400cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb745557640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7400cd280 0x7fb7400cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44782/0 (socket says 172.21.15.181:44782) 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.180+0000 7fb745557640 1 -- 172.21.15.181:0/2936605688 learned_addr learned my addr 172.21.15.181:0/2936605688 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb744d56640 1 --2- 172.21.15.181:0/2936605688 >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 0x7fb7400d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb745557640 1 -- 172.21.15.181:0/2936605688 >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 msgr2=0x7fb7400d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb745557640 1 --2- 172.21.15.181:0/2936605688 >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 0x7fb7400d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.169 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb745557640 1 -- 172.21.15.181:0/2936605688 >> v1:172.21.15.187:6789/0 conn(0x7fb7400cc9d0 legacy=0x7fb74015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb745557640 1 -- 172.21.15.181:0/2936605688 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7400cf9a0 con 0x7fb7400cd280 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb744d56640 1 --2- 172.21.15.181:0/2936605688 >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 0x7fb7400d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.181+0000 7fb745557640 1 --2- 172.21.15.181:0/2936605688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cd280 0x7fb7400cd6d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb7400673e0 tx=0x7fb73802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7d9a59784c3149 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb73f7fe640 1 -- 172.21.15.181:0/2936605688 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73802ca80 con 0x7fb7400cd280 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb73f7fe640 1 -- 172.21.15.181:0/2936605688 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73802cc20 con 0x7fb7400cd280 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb73f7fe640 1 -- 172.21.15.181:0/2936605688 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7380343f0 con 0x7fb7400cd280 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 -- 172.21.15.181:0/2936605688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cd280 msgr2=0x7fb7400cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.170 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 --2- 172.21.15.181:0/2936605688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cd280 0x7fb7400cd6d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb7400673e0 tx=0x7fb73802bee0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 -- 172.21.15.181:0/2936605688 shutdown_connections 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 --2- 172.21.15.181:0/2936605688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400cd280 0x7fb7400cd6d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 --2- 172.21.15.181:0/2936605688 >> v2:172.21.15.187:3300/0 conn(0x7fb7400af990 0x7fb7400d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 -- 172.21.15.181:0/2936605688 >> 172.21.15.181:0/2936605688 conn(0x7fb7400d0870 msgr2=0x7fb7400d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 -- 172.21.15.181:0/2936605688 shutdown_connections 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.182+0000 7fb746fba640 1 -- 172.21.15.181:0/2936605688 wait complete. 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.183+0000 7fb746fba640 1 Processor -- start 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.183+0000 7fb746fba640 1 -- start start 2024-03-29T04:11:39.171 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 0x7fb74014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7400cc9d0 0x7fb740152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7400cd280 0x7fb740153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400cf480 con 0x7fb7400af990 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400cf300 con 0x7fb7400cd280 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb746fba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400afd70 con 0x7fb7400cc9d0 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb744d56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 0x7fb74014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb744d56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 0x7fb74014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44792/0 (socket says 172.21.15.181:44792) 2024-03-29T04:11:39.172 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb744d56640 1 -- 172.21.15.181:0/1566264370 learned_addr learned my addr 172.21.15.181:0/1566264370 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb745557640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7400cd280 0x7fb740153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb73ffff640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7400cc9d0 0x7fb740152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.184+0000 7fb744d56640 1 -- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7400cc9d0 msgr2=0x7fb740152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb744d56640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7400cc9d0 0x7fb740152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb744d56640 1 -- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7400cd280 msgr2=0x7fb740153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb744d56640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7400cd280 0x7fb740153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb744d56640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74014e3e0 con 0x7fb7400af990 2024-03-29T04:11:39.173 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb73ffff640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7400cc9d0 0x7fb740152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb745557640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7400cd280 0x7fb740153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb744d56640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 0x7fb74014fdd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c7f0 tx=0x7fb73000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730015070 con 0x7fb7400af990 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb730007820 con 0x7fb7400af990 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74014e6c0 con 0x7fb7400af990 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730007bd0 con 0x7fb7400af990 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.185+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74014ec80 con 0x7fb7400af990 2024-03-29T04:11:39.174 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.186+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb740075900 con 0x7fb7400af990 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.186+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb730018070 con 0x7fb7400af990 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73d7fa640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb72c06c530 0x7fb72c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb73008b6c0 con 0x7fb7400af990 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73ffff640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb72c06c530 0x7fb72c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73d7fa640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7fb72c0729d0 0x7fb72c074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb73001de50 con 0x7fb72c0729d0 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb730092050 con 0x7fb7400af990 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb73ffff640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb72c06c530 0x7fb72c06e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb74014fb50 tx=0x7fb734009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.175 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb745557640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7fb72c0729d0 0x7fb72c074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.187+0000 7fb745557640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7fb72c0729d0 0x7fb72c074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.176 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.188+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== osd.5 v2:172.21.15.187:6820/461978944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb73001de50 con 0x7fb72c0729d0 2024-03-29T04:11:39.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.201+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 --> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb740150b20 con 0x7fb72c0729d0 2024-03-29T04:11:39.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb73d7fa640 1 -- 172.21.15.181:0/1566264370 <== osd.5 v2:172.21.15.187:6820/461978944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb740150b20 con 0x7fb72c0729d0 2024-03-29T04:11:39.182 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:39.182 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:39.182 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7fb72c0729d0 msgr2=0x7fb72c074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.187:6820/461978944,v1:172.21.15.187:6822/461978944] conn(0x7fb72c0729d0 0x7fb72c074e10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb72c06c530 msgr2=0x7fb72c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fb72c06c530 0x7fb72c06e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb74014fb50 tx=0x7fb734009040 comp rx=0 tx=0).stop 2024-03-29T04:11:39.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 msgr2=0x7fb74014fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 --2- 172.21.15.181:0/1566264370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7400af990 0x7fb74014fdd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c7f0 tx=0x7fb73000ccb0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb745557640 1 -- 172.21.15.181:0/1566264370 reap_dead start 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 shutdown_connections 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 >> 172.21.15.181:0/1566264370 conn(0x7fb7400d0870 msgr2=0x7fb7400b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 shutdown_connections 2024-03-29T04:11:39.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.202+0000 7fb746fba640 1 -- 172.21.15.181:0/1566264370 wait complete. 2024-03-29T04:11:39.191 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T04:11:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.313+0000 7f3cc6c4e640 1 Processor -- start 2024-03-29T04:11:39.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 -- start start 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 0x7f3cc00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 0x7f3cc00d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc00d3120 con 0x7f3cc015e7a0 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc00cfc40 con 0x7f3cc00cec60 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc6c4e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc00d3400 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.295 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc49ea640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f3cc015e7a0 0x7f3cc00cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:53986/0 (socket says 172.21.15.181:53986) 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.314+0000 7f3cc49ea640 1 -- 172.21.15.181:0/2586956851 learned_addr learned my addr 172.21.15.181:0/2586956851 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/2586956851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 0x7f3cc00d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cbffff640 1 --2- 172.21.15.181:0/2586956851 >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 0x7f3cc00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372151374 0 0) 0x7f3cc00d3120 con 0x7f3cc015e7a0 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8003520 con 0x7f3cc015e7a0 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 <== mon.0 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1349672820 0 0) 0x7f3ca8003520 con 0x7f3cc015e7a0 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cc51eb640 1 -- 172.21.15.181:0/2586956851 >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 msgr2=0x7f3cc00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/2586956851 >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 0x7f3cc00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.296 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cc51eb640 1 -- 172.21.15.181:0/2586956851 >> v1:172.21.15.181:6789/0 conn(0x7f3cc015e7a0 legacy=0x7f3cc00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cc51eb640 1 -- 172.21.15.181:0/2586956851 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc00d36e0 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.315+0000 7f3cbffff640 1 --2- 172.21.15.181:0/2586956851 >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 0x7f3cc00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/2586956851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 0x7f3cc00d17f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009d90 tx=0x7f3cb802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebd71d674077287 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8039070 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb802c810 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.297 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cbf7fe640 1 -- 172.21.15.181:0/2586956851 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0001670 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/2586956851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 msgr2=0x7f3cc00d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.316+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/2586956851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 0x7f3cc00d17f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009d90 tx=0x7f3cb802bce0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/2586956851 shutdown_connections 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/2586956851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00d3ee0 0x7f3cc00d17f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/2586956851 >> v2:172.21.15.187:3300/0 conn(0x7f3cc00cec60 0x7f3cc00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.300 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/2586956851 >> 172.21.15.181:0/2586956851 conn(0x7f3cc00676b0 msgr2=0x7f3cc0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/2586956851 shutdown_connections 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.317+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/2586956851 wait complete. 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.318+0000 7f3cc6c4e640 1 Processor -- start 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.318+0000 7f3cc6c4e640 1 -- start start 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.318+0000 7f3cc6c4e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00cec60 0x7f3cc0158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cc00d3ee0 0x7f3cc0160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.301 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00b7720 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00b75a0 con 0x7f3cc00d3ee0 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc00b78a0 con 0x7f3cc00cec60 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44798/0 (socket says 172.21.15.181:44798) 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 -- 172.21.15.181:0/1749809221 learned_addr learned my addr 172.21.15.181:0/1749809221 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cbffff640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cc00d3ee0 0x7f3cc0160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc49ea640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00cec60 0x7f3cc0158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 -- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00cec60 msgr2=0x7f3cc0158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.302 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00cec60 0x7f3cc0158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 -- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cc00d3ee0 msgr2=0x7f3cc0160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cc00d3ee0 0x7f3cc0160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 -- 172.21.15.181:0/1749809221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc00b6280 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc51eb640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009eb0 tx=0x7f3cb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8039050 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8045070 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc00b6560 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.303 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8009450 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.319+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc00b3ce0 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.320+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc00b3340 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.304 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.321+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3cb8040690 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.321+0000 7f3cbd7fa640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 0x7f3cac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.321+0000 7f3cc49ea640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 0x7f3cac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.321+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f3cb80ad980 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.322+0000 7f3cc49ea640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 0x7f3cac06e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00017e0 tx=0x7f3cb0001570 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.305 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.326+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3cb804f020 con 0x7f3cc00cdfe0 2024-03-29T04:11:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.504+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3cc00b3eb0 con 0x7f3cac06c530 2024-03-29T04:11:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.505+0000 7f3cbd7fa640 1 -- 172.21.15.181:0/1749809221 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3cc00b3eb0 con 0x7f3cac06c530 2024-03-29T04:11:39.484 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T04:11:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 msgr2=0x7f3cac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 0x7f3cac06e9f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00017e0 tx=0x7f3cb0001570 comp rx=0 tx=0).stop 2024-03-29T04:11:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 msgr2=0x7f3cc00b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009eb0 tx=0x7f3cb8004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 shutdown_connections 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cc00cdfe0 0x7f3cc00b5910 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cc00d3ee0 0x7f3cc0160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f3cac06c530 0x7f3cac06e9f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 --2- 172.21.15.181:0/1749809221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3cc00cec60 0x7f3cc0158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 >> 172.21.15.181:0/1749809221 conn(0x7f3cc00676b0 msgr2=0x7f3cc00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 shutdown_connections 2024-03-29T04:11:39.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.507+0000 7f3cc6c4e640 1 -- 172.21.15.181:0/1749809221 wait complete. 2024-03-29T04:11:39.497 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T04:11:39.497 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:39.602 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.623+0000 7fe7b5a37640 1 Processor -- start 2024-03-29T04:11:39.603 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.623+0000 7fe7b5a37640 1 -- start start 2024-03-29T04:11:39.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7b5a37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b015e440 0x7fe7b00b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.604 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7b5a37640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe7b006d4a0 0x7fe7b00d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7b5a37640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b00d2910 con 0x7fe7b00d3ef0 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7b5a37640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b00d3520 con 0x7fe7b006d4a0 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7b5a37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b00d2bf0 con 0x7fe7b015e440 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe7b006d4a0 0x7fe7b00d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe7b006d4a0 0x7fe7b00d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44802/0 (socket says 172.21.15.181:44802) 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/686565892 learned_addr learned my addr 172.21.15.181:0/686565892 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7aeffd640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b015e440 0x7fe7b00b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.605 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/686565892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b015e440 msgr2=0x7fe7b00b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b015e440 0x7fe7b00b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/686565892 >> v1:172.21.15.187:6789/0 conn(0x7fe7b00d3ef0 legacy=0x7fe7b00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.624+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/686565892 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b00d2ed0 con 0x7fe7b006d4a0 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.625+0000 7fe7ae7fc640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b006d4a0 0x7fe7b00d39b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009870 tx=0x7fe7a402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0876f5ed5279752 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.625+0000 7fe7adffb640 1 -- 172.21.15.181:0/686565892 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a402f840 con 0x7fe7b006d4a0 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.625+0000 7fe7adffb640 1 -- 172.21.15.181:0/686565892 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a402f9e0 con 0x7fe7b006d4a0 2024-03-29T04:11:39.606 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.625+0000 7fe7adffb640 1 -- 172.21.15.181:0/686565892 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a402fdb0 con 0x7fe7b006d4a0 2024-03-29T04:11:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 -- 172.21.15.181:0/686565892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b006d4a0 msgr2=0x7fe7b00d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b006d4a0 0x7fe7b00d39b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009870 tx=0x7fe7a402ba20 comp rx=0 tx=0).stop 2024-03-29T04:11:39.610 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 -- 172.21.15.181:0/686565892 shutdown_connections 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b006d4a0 0x7fe7b00d39b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/686565892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b015e440 0x7fe7b00b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 -- 172.21.15.181:0/686565892 >> 172.21.15.181:0/686565892 conn(0x7fe7b00676b0 msgr2=0x7fe7b0067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 -- 172.21.15.181:0/686565892 shutdown_connections 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.626+0000 7fe7b5a37640 1 -- 172.21.15.181:0/686565892 wait complete. 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.627+0000 7fe7b5a37640 1 Processor -- start 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.627+0000 7fe7b5a37640 1 -- start start 2024-03-29T04:11:39.611 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b006d4a0 0x7fe7b01495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 0x7fe7b0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7b0156780 0x7fe7b01610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b015e820 con 0x7fe7b015e440 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b00ba220 con 0x7fe7b0156780 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7b5a37640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b015e9a0 con 0x7fe7b006d4a0 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 0x7fe7b0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 0x7fe7b0156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44810/0 (socket says 172.21.15.181:44810) 2024-03-29T04:11:39.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/1023763798 learned_addr learned my addr 172.21.15.181:0/1023763798 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7aeffd640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b006d4a0 0x7fe7b01495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7af7fe640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7b0156780 0x7fe7b01610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b006d4a0 msgr2=0x7fe7b01495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7b006d4a0 0x7fe7b01495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7b0156780 msgr2=0x7fe7b01610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7b0156780 0x7fe7b01610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b0149b00 con 0x7fe7b015e440 2024-03-29T04:11:39.613 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.628+0000 7fe7ae7fc640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 0x7fe7b0156240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009870 tx=0x7fe7a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.629+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4037d10 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.629+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a4046070 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.629+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b0158390 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.629+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4041640 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.629+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b0158950 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7b0075900 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe7a4040070 con 0x7fe7b015e440 2024-03-29T04:11:39.614 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe797fff640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe79c06c530 0x7fe79c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fe7a40adbc0 con 0x7fe7b015e440 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe7aeffd640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe79c06c530 0x7fe79c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe797fff640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fe79c0729d0 0x7fe79c074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.630+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7a4008f30 con 0x7fe79c0729d0 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.631+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe7a40b4020 con 0x7fe7b015e440 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.631+0000 7fe7aeffd640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe79c06c530 0x7fe79c06e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0152c70 tx=0x7fe7a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.631+0000 7fe7af7fe640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fe79c0729d0 0x7fe79c074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.631+0000 7fe7af7fe640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fe79c0729d0 0x7fe79c074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.632+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== osd.6 v2:172.21.15.187:6808/3602434128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7a4008f30 con 0x7fe79c0729d0 2024-03-29T04:11:39.624 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.645+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 --> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe7b00c8690 con 0x7fe79c0729d0 2024-03-29T04:11:39.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe797fff640 1 -- 172.21.15.181:0/1023763798 <== osd.6 v2:172.21.15.187:6808/3602434128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe7b00c8690 con 0x7fe79c0729d0 2024-03-29T04:11:39.625 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:39.625 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:39.625 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fe79c0729d0 msgr2=0x7fe79c074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.187:6808/3602434128,v1:172.21.15.187:6809/3602434128] conn(0x7fe79c0729d0 0x7fe79c074e10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe79c06c530 msgr2=0x7fe79c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fe79c06c530 0x7fe79c06e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0152c70 tx=0x7fe7a0008040 comp rx=0 tx=0).stop 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 msgr2=0x7fe7b0156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 --2- 172.21.15.181:0/1023763798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7b015e440 0x7fe7b0156240 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009870 tx=0x7fe7a4004730 comp rx=0 tx=0).stop 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7af7fe640 1 -- 172.21.15.181:0/1023763798 reap_dead start 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 shutdown_connections 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 >> 172.21.15.181:0/1023763798 conn(0x7fe7b00676b0 msgr2=0x7fe7b00b8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 shutdown_connections 2024-03-29T04:11:39.627 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.646+0000 7fe7b5a37640 1 -- 172.21.15.181:0/1023763798 wait complete. 2024-03-29T04:11:39.636 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T04:11:39.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.758+0000 7fd8a2be2640 1 Processor -- start 2024-03-29T04:11:39.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.758+0000 7fd8a2be2640 1 -- start start 2024-03-29T04:11:39.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a2be2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a2be2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c15feb0 con 0x7fd89c15f6c0 2024-03-29T04:11:39.739 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a2be2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c15c910 con 0x7fd89c0cb050 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a2be2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c15c480 con 0x7fd89c0cbed0 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a117f640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd89c15f6c0 0x7fd89c151c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:54000/0 (socket says 172.21.15.181:54000) 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd8a117f640 1 -- 172.21.15.181:0/2210152610 learned_addr learned my addr 172.21.15.181:0/2210152610 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89bfff640 1 --2- 172.21.15.181:0/2210152610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639281092 0 0) 0x7fd89c15feb0 con 0x7fd89c15f6c0 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd88c003520 con 0x7fd89c15f6c0 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215407052 0 0) 0x7fd89c15c910 con 0x7fd89c0cb050 2024-03-29T04:11:39.740 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c15feb0 con 0x7fd89c0cb050 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89bfff640 1 -- 172.21.15.181:0/2210152610 >> v1:172.21.15.187:6789/0 conn(0x7fd89c0cb050 legacy=0x7fd89c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89bfff640 1 -- 172.21.15.181:0/2210152610 >> v1:172.21.15.181:6789/0 conn(0x7fd89c15f6c0 legacy=0x7fd89c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.759+0000 7fd89bfff640 1 -- 172.21.15.181:0/2210152610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c15bb50 con 0x7fd89c0cbed0 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.760+0000 7fd89bfff640 1 --2- 172.21.15.181:0/2210152610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c15f180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8880097d0 tx=0x7fd88802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a714477b2c735320 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.760+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035020 con 0x7fd89c0cbed0 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.760+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd888031890 con 0x7fd89c0cbed0 2024-03-29T04:11:39.741 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.760+0000 7fd89b7fe640 1 -- 172.21.15.181:0/2210152610 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888031c60 con 0x7fd89c0cbed0 2024-03-29T04:11:39.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 -- 172.21.15.181:0/2210152610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 msgr2=0x7fd89c15f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/2210152610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c15f180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8880097d0 tx=0x7fd88802b930 comp rx=0 tx=0).stop 2024-03-29T04:11:39.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 -- 172.21.15.181:0/2210152610 shutdown_connections 2024-03-29T04:11:39.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/2210152610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c15f180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.744 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 -- 172.21.15.181:0/2210152610 >> 172.21.15.181:0/2210152610 conn(0x7fd89c06d610 msgr2=0x7fd89c06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 -- 172.21.15.181:0/2210152610 shutdown_connections 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.761+0000 7fd8a2be2640 1 -- 172.21.15.181:0/2210152610 wait complete. 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.762+0000 7fd8a2be2640 1 Processor -- start 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.762+0000 7fd8a2be2640 1 -- start start 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6c0 0x7fd89c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c0d08b0 con 0x7fd89c0cb050 2024-03-29T04:11:39.745 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c15c110 con 0x7fd89c15f6c0 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a2be2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c15fb70 con 0x7fd89c0cbed0 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44820/0 (socket says 172.21.15.181:44820) 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 -- 172.21.15.181:0/3008128759 learned_addr learned my addr 172.21.15.181:0/3008128759 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a117f640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6c0 0x7fd89c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd89bfff640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 -- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 msgr2=0x7fd89c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.746 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 -- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6c0 msgr2=0x7fd89c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6c0 0x7fd89c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 -- 172.21.15.181:0/3008128759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c154440 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.763+0000 7fd8a097e640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd89000c500 tx=0x7fd89000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.764+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8900052d0 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.764+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd890005490 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.764+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c154720 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.764+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890012870 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.764+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c154fc0 con 0x7fd89c0cb050 2024-03-29T04:11:39.747 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.765+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c1548f0 con 0x7fd89c0cb050 2024-03-29T04:11:39.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.765+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd890012a10 con 0x7fd89c0cb050 2024-03-29T04:11:39.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.765+0000 7fd8997fa640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 0x7fd88006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:39.748 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.765+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fd89008b0e0 con 0x7fd89c0cb050 2024-03-29T04:11:39.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.765+0000 7fd89bfff640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 0x7fd88006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:39.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.766+0000 7fd89bfff640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 0x7fd88006e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd888036000 tx=0x7fd888008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:39.749 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.770+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd890059230 con 0x7fd89c0cb050 2024-03-29T04:11:39.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.945+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd89c075900 con 0x7fd88006c530 2024-03-29T04:11:39.924 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.945+0000 7fd8997fa640 1 -- 172.21.15.181:0/3008128759 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd89c075900 con 0x7fd88006c530 2024-03-29T04:11:39.925 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T04:11:39.926 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 msgr2=0x7fd88006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 0x7fd88006e9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd888036000 tx=0x7fd888008030 comp rx=0 tx=0).stop 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 msgr2=0x7fd89c14b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd89000c500 tx=0x7fd89000c9c0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 shutdown_connections 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd89c15f6c0 0x7fd89c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd88006c530 0x7fd88006e9f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.927 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd89c0cbed0 0x7fd89c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 --2- 172.21.15.181:0/3008128759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd89c0cb050 0x7fd89c14b370 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 >> 172.21.15.181:0/3008128759 conn(0x7fd89c06d610 msgr2=0x7fd89c0cb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 shutdown_connections 2024-03-29T04:11:39.928 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:39.947+0000 7fd8a2be2640 1 -- 172.21.15.181:0/3008128759 wait complete. 2024-03-29T04:11:39.937 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T04:11:39.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T04:11:40.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.063+0000 7f04899ca640 1 Processor -- start 2024-03-29T04:11:40.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.063+0000 7f04899ca640 1 -- start start 2024-03-29T04:11:40.044 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04899ca640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f048415ead0 0x7f048415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04899ca640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f048406d310 0x7f048406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04899ca640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f048415f850 con 0x7f04840d2110 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04899ca640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f048406c830 con 0x7f048406d310 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04899ca640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f048406cb10 con 0x7f048415ead0 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f048406d310 0x7f048406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f048406d310 0x7f048406d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44822/0 (socket says 172.21.15.181:44822) 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 -- 172.21.15.181:0/3125959247 learned_addr learned my addr 172.21.15.181:0/3125959247 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.045 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04827fc640 1 --2- 172.21.15.181:0/3125959247 >> v2:172.21.15.187:3300/0 conn(0x7f048415ead0 0x7f048415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 -- 172.21.15.181:0/3125959247 >> v2:172.21.15.187:3300/0 conn(0x7f048415ead0 msgr2=0x7f048415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 --2- 172.21.15.181:0/3125959247 >> v2:172.21.15.187:3300/0 conn(0x7f048415ead0 0x7f048415ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 -- 172.21.15.181:0/3125959247 >> v1:172.21.15.187:6789/0 conn(0x7f04840d2110 legacy=0x7f04840d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.064+0000 7f04837fe640 1 -- 172.21.15.181:0/3125959247 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048406cdf0 con 0x7f048406d310 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.065+0000 7f04837fe640 1 --2- 172.21.15.181:0/3125959247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048406d780 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0484068190 tx=0x7f047c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59736237e6d2bee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.065+0000 7f0481ffb640 1 -- 172.21.15.181:0/3125959247 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c035020 con 0x7f048406d310 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.065+0000 7f0481ffb640 1 -- 172.21.15.181:0/3125959247 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047c031870 con 0x7f048406d310 2024-03-29T04:11:40.046 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.065+0000 7f0481ffb640 1 -- 172.21.15.181:0/3125959247 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c031c40 con 0x7f048406d310 2024-03-29T04:11:40.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 -- 172.21.15.181:0/3125959247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 msgr2=0x7f048406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 --2- 172.21.15.181:0/3125959247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048406d780 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0484068190 tx=0x7f047c02b910 comp rx=0 tx=0).stop 2024-03-29T04:11:40.050 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 -- 172.21.15.181:0/3125959247 shutdown_connections 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 --2- 172.21.15.181:0/3125959247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048406d780 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 --2- 172.21.15.181:0/3125959247 >> v2:172.21.15.187:3300/0 conn(0x7f048415ead0 0x7f048415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 -- 172.21.15.181:0/3125959247 >> 172.21.15.181:0/3125959247 conn(0x7f04840672c0 msgr2=0x7f04840676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 -- 172.21.15.181:0/3125959247 shutdown_connections 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.066+0000 7f04899ca640 1 -- 172.21.15.181:0/3125959247 wait complete. 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.067+0000 7f04899ca640 1 Processor -- start 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.067+0000 7f04899ca640 1 -- start start 2024-03-29T04:11:40.051 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04840d2110 0x7f04840cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f048415ead0 0x7f04840b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f048415c510 con 0x7f048406d310 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f048415c390 con 0x7f04840d2110 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04899ca640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f048406dcc0 con 0x7f048415ead0 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048414a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44828/0 (socket says 172.21.15.181:44828) 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 -- 172.21.15.181:0/3717643139 learned_addr learned my addr 172.21.15.181:0/3717643139 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04827fc640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04840d2110 0x7f04840cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04837fe640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f048415ead0 0x7f04840b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 -- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f048415ead0 msgr2=0x7f04840b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f048415ead0 0x7f04840b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 -- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04840d2110 msgr2=0x7f04840cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04840d2110 0x7f04840cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f0482ffd640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04840cdac0 con 0x7f048406d310 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.068+0000 7f04837fe640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f048415ead0 0x7f04840b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:40.053 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f0482ffd640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048414a6a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f047400c7a0 tx=0x7f047400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474017070 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474003da0 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04840cdda0 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740052b0 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.069+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04840ce360 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.070+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0484075900 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.070+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f047401a070 con 0x7f048406d310 2024-03-29T04:11:40.054 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.070+0000 7f046bfff640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f047006c530 0x7f047006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.070+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f047408ab30 con 0x7f048406d310 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.070+0000 7f04827fc640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f047006c530 0x7f047006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f046bfff640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7f04700729d0 0x7f0470074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04700754e0 con 0x7f04700729d0 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f047408f2c0 con 0x7f048406d310 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f04837fe640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7f04700729d0 0x7f0470074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f04827fc640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f047006c530 0x7f047006e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0484162ce0 tx=0x7f0478008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.055 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.071+0000 7f04837fe640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7f04700729d0 0x7f0470074e10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.056 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.072+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== osd.7 v2:172.21.15.187:6815/235935836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04700754e0 con 0x7f04700729d0 2024-03-29T04:11:40.064 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.085+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 --> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f04840702c0 con 0x7f04700729d0 2024-03-29T04:11:40.065 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f046bfff640 1 -- 172.21.15.181:0/3717643139 <== osd.7 v2:172.21.15.187:6815/235935836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f04840702c0 con 0x7f04700729d0 2024-03-29T04:11:40.065 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-03-29T04:11:40.065 INFO:teuthology.orchestra.run.smithi181.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T04:11:40.065 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-03-29T04:11:40.066 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7f04700729d0 msgr2=0x7f0470074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.187:6815/235935836,v1:172.21.15.187:6816/235935836] conn(0x7f04700729d0 0x7f0470074e10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f047006c530 msgr2=0x7f047006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f047006c530 0x7f047006e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0484162ce0 tx=0x7f0478008040 comp rx=0 tx=0).stop 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 msgr2=0x7f048414a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 --2- 172.21.15.181:0/3717643139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f048406d310 0x7f048414a6a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f047400c7a0 tx=0x7f047400cc60 comp rx=0 tx=0).stop 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04837fe640 1 -- 172.21.15.181:0/3717643139 reap_dead start 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 shutdown_connections 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 >> 172.21.15.181:0/3717643139 conn(0x7f04840672c0 msgr2=0x7f04840676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 shutdown_connections 2024-03-29T04:11:40.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.086+0000 7f04899ca640 1 -- 172.21.15.181:0/3717643139 wait complete. 2024-03-29T04:11:40.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T04:11:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.198+0000 7fc1e487b640 1 Processor -- start 2024-03-29T04:11:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.198+0000 7fc1e487b640 1 -- start start 2024-03-29T04:11:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1e487b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.181 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1e487b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1e015cbe0 0x7fc1e015d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1e487b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e015dbb0 con 0x7fc1e00ccd90 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1e487b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e00b46c0 con 0x7fc1e015cbe0 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1e487b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e015d8d0 con 0x7fc1e00b3d20 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1ded9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1e015cbe0 0x7fc1e015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1ded9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1e015cbe0 0x7fc1e015d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44842/0 (socket says 172.21.15.181:44842) 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.199+0000 7fc1ded9d640 1 -- 172.21.15.181:0/1988720624 learned_addr learned my addr 172.21.15.181:0/1988720624 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1d7fff640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ded9d640 1 -- 172.21.15.181:0/1988720624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 msgr2=0x7fc1e00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:40.183 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ded9d640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e00b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ded9d640 1 -- 172.21.15.181:0/1988720624 >> v1:172.21.15.187:6789/0 conn(0x7fc1e00ccd90 legacy=0x7fc1e00d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ded9d640 1 -- 172.21.15.181:0/1988720624 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e00d0090 con 0x7fc1e015cbe0 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1d7fff640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ded9d640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e015d030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009e50 tx=0x7fc1d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae01d4153c10498 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.200+0000 7fc1ddd9b640 1 -- 172.21.15.181:0/1988720624 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d802ca70 con 0x7fc1e015cbe0 2024-03-29T04:11:40.184 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1ddd9b640 1 -- 172.21.15.181:0/1988720624 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d802cc10 con 0x7fc1e015cbe0 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1ddd9b640 1 -- 172.21.15.181:0/1988720624 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80343f0 con 0x7fc1e015cbe0 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 -- 172.21.15.181:0/1988720624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 msgr2=0x7fc1e015d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e015d030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009e50 tx=0x7fc1d802bed0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 -- 172.21.15.181:0/1988720624 shutdown_connections 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e015d030 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 --2- 172.21.15.181:0/1988720624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 -- 172.21.15.181:0/1988720624 >> 172.21.15.181:0/1988720624 conn(0x7fc1e00672c0 msgr2=0x7fc1e00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 -- 172.21.15.181:0/1988720624 shutdown_connections 2024-03-29T04:11:40.185 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.201+0000 7fc1e487b640 1 -- 172.21.15.181:0/1988720624 wait complete. 2024-03-29T04:11:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.202+0000 7fc1e487b640 1 Processor -- start 2024-03-29T04:11:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.202+0000 7fc1e487b640 1 -- start start 2024-03-29T04:11:40.188 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e01e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e00ccd90 0x7fc1e0149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e015d570 con 0x7fc1e015cbe0 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e00d0650 con 0x7fc1e00ccd90 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1e487b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e00cd170 con 0x7fc1e00b3d20 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1ded9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1ded9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44846/0 (socket says 172.21.15.181:44846) 2024-03-29T04:11:40.189 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1ded9d640 1 -- 172.21.15.181:0/3232849690 learned_addr learned my addr 172.21.15.181:0/3232849690 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1d7fff640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e00ccd90 0x7fc1e0149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.203+0000 7fc1de59c640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e01e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 -- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 msgr2=0x7fc1e01e4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e01e4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 -- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e00ccd90 msgr2=0x7fc1e0149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e00ccd90 0x7fc1e0149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 -- 172.21.15.181:0/3232849690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e00c8330 con 0x7fc1e015cbe0 2024-03-29T04:11:40.190 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1ded9d640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8036040 tx=0x7fc1d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8043020 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d803e450 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8009510 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e00c9050 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.204+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e00c7850 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.205+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0074050 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.205+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc1d8046020 con 0x7fc1e015cbe0 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.206+0000 7fc1d6ffd640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 0x7fc1c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.191 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.206+0000 7fc1de59c640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 0x7fc1c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.206+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc1d80adbb0 con 0x7fc1e015cbe0 2024-03-29T04:11:40.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.206+0000 7fc1de59c640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 0x7fc1c006e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0001780 tx=0x7fc1d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.192 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.210+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc1d804d030 con 0x7fc1e015cbe0 2024-03-29T04:11:40.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.383+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fc1e0075900 con 0x7fc1c006c530 2024-03-29T04:11:40.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.384+0000 7fc1d6ffd640 1 -- 172.21.15.181:0/3232849690 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc1e0075900 con 0x7fc1c006c530 2024-03-29T04:11:40.363 INFO:teuthology.orchestra.run.smithi181.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T04:11:40.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 msgr2=0x7fc1c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.365 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 0x7fc1c006e9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0001780 tx=0x7fc1d0009040 comp rx=0 tx=0).stop 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 msgr2=0x7fc1e00b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8036040 tx=0x7fc1d8004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 shutdown_connections 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e015cbe0 0x7fc1e00b6fa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1e00ccd90 0x7fc1e0149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fc1c006c530 0x7fc1c006e9f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.366 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 --2- 172.21.15.181:0/3232849690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1e00b3d20 0x7fc1e01e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 >> 172.21.15.181:0/3232849690 conn(0x7fc1e00672c0 msgr2=0x7fc1e00d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 shutdown_connections 2024-03-29T04:11:40.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.386+0000 7fc1e487b640 1 -- 172.21.15.181:0/3232849690 wait complete. 2024-03-29T04:11:40.376 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:11:40.477 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.498+0000 7fd217e35640 1 Processor -- start 2024-03-29T04:11:40.478 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 -- start start 2024-03-29T04:11:40.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.479 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2100d36f0 con 0x7fd2100d48c0 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd2100d3cb0 con 0x7fd2100d30d0 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.499+0000 7fd217e35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd2100d4d00 con 0x7fd21015e950 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd2163d2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd2163d2640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.181:50912/0 (socket says 172.21.15.181:50912) 2024-03-29T04:11:40.480 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd2153d0640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fd2100d48c0 0x7fd2100d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.181:45240/0 (socket says 172.21.15.181:45240) 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd2163d2640 1 -- 172.21.15.181:0/918168219 learned_addr learned my addr 172.21.15.181:0/918168219 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.181:46914/0 (socket says 172.21.15.181:46914) 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 -- 172.21.15.181:0/918168219 >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 msgr2=0x7fd2100b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 --2- 172.21.15.181:0/918168219 >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 -- 172.21.15.181:0/918168219 >> v1:172.21.15.187:6789/0 conn(0x7fd2100d48c0 legacy=0x7fd2100d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd215bd1640 1 -- 172.21.15.181:0/918168219 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2100d39d0 con 0x7fd21015e950 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd214bcf640 1 -- 172.21.15.181:0/918168219 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5991850 0 0) 0x7fd2100d36f0 con 0x7fd2100d48c0 2024-03-29T04:11:40.481 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.500+0000 7fd2163d2640 1 --2- 172.21.15.181:0/918168219 >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T04:11:40.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd215bd1640 1 --2- 172.21.15.181:0/918168219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf214230647f5879 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd214bcf640 1 -- 172.21.15.181:0/918168219 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208039070 con 0x7fd21015e950 2024-03-29T04:11:40.485 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd214bcf640 1 -- 172.21.15.181:0/918168219 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20802cce0 con 0x7fd21015e950 2024-03-29T04:11:40.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd214bcf640 1 -- 172.21.15.181:0/918168219 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208034660 con 0x7fd21015e950 2024-03-29T04:11:40.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd217e35640 1 -- 172.21.15.181:0/918168219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 msgr2=0x7fd2100d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.501+0000 7fd217e35640 1 --2- 172.21.15.181:0/918168219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20802bf30 comp rx=0 tx=0).stop 2024-03-29T04:11:40.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 -- 172.21.15.181:0/918168219 shutdown_connections 2024-03-29T04:11:40.486 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 --2- 172.21.15.181:0/918168219 >> v2:172.21.15.187:3300/0 conn(0x7fd2100d30d0 0x7fd2100b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 --2- 172.21.15.181:0/918168219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd21015e950 0x7fd2100d4380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 -- 172.21.15.181:0/918168219 >> 172.21.15.181:0/918168219 conn(0x7fd2100676e0 msgr2=0x7fd210067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 -- 172.21.15.181:0/918168219 shutdown_connections 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.502+0000 7fd217e35640 1 -- 172.21.15.181:0/918168219 wait complete. 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 Processor -- start 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 -- start start 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2100d48c0 0x7fd2101582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.487 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd21015e950 0x7fd2100cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd21015ed30 con 0x7fd2100d30d0 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100b7da0 con 0x7fd21015e950 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.503+0000 7fd217e35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd21015eeb0 con 0x7fd2100d48c0 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:44852/0 (socket says 172.21.15.181:44852) 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 -- 172.21.15.181:0/887131714 learned_addr learned my addr 172.21.15.181:0/887131714 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd2153d0640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2100d48c0 0x7fd2101582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd2163d2640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd21015e950 0x7fd2100cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 -- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2100d48c0 msgr2=0x7fd2101582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2100d48c0 0x7fd2101582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 -- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd21015e950 msgr2=0x7fd2100cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd21015e950 0x7fd2100cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 -- 172.21.15.181:0/887131714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd210158e00 con 0x7fd2100d30d0 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd215bd1640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd208037040 tx=0x7fd208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208039050 con 0x7fd2100d30d0 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20803a040 con 0x7fd2100d30d0 2024-03-29T04:11:40.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd210158b20 con 0x7fd2100d30d0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208009450 con 0x7fd2100d30d0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.504+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21015a7c0 con 0x7fd2100d30d0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.505+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd210075a10 con 0x7fd2100d30d0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.506+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd208047020 con 0x7fd2100d30d0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.506+0000 7fd20e7fc640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 0x7fd1fc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.506+0000 7fd2153d0640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 0x7fd1fc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:11:40.490 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.506+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fd2080ad5e0 con 0x7fd2100d30d0 2024-03-29T04:11:40.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.507+0000 7fd2153d0640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 0x7fd1fc06ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2100cc1d0 tx=0x7fd204007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:11:40.491 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.510+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd20804e020 con 0x7fd2100d30d0 2024-03-29T04:11:40.662 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.683+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd21015dbb0 con 0x7fd1fc06c560 2024-03-29T04:11:40.663 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.684+0000 7fd20e7fc640 1 -- 172.21.15.181:0/887131714 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13406 (secure 0 0 0) 0x7fd21015dbb0 con 0x7fd1fc06c560 2024-03-29T04:11:40.663 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:11:40.664 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:11:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 msgr2=0x7fd1fc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 0x7fd1fc06ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2100cc1d0 tx=0x7fd204007d50 comp rx=0 tx=0).stop 2024-03-29T04:11:40.665 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 msgr2=0x7fd21015afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd208037040 tx=0x7fd208004a40 comp rx=0 tx=0).stop 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 shutdown_connections 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd21015e950 0x7fd2100cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7fd1fc06c560 0x7fd1fc06ea20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2100d48c0 0x7fd2101582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 --2- 172.21.15.181:0/887131714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2100d30d0 0x7fd21015afc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:11:40.666 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 >> 172.21.15.181:0/887131714 conn(0x7fd2100676e0 msgr2=0x7fd2100b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:11:40.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 shutdown_connections 2024-03-29T04:11:40.667 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:11:40.686+0000 7fd217e35640 1 -- 172.21.15.181:0/887131714 wait complete. 2024-03-29T04:11:40.676 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-03-29T04:11:39.600416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225584,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749136,"statfs":{"total":773094113280,"available":772863115264,"internally_reserved":0,"allocated":5349376,"data_stored":3010896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173890432,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-75,"num_read_kb":0,"num_write":-193,"num_write_kb":-169854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.149077"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-29T04:11:32.458481+0000","last_change":"2024-03-29T04:09:04.724674+0000","last_active":"2024-03-29T04:11:32.458481+0000","last_peered":"2024-03-29T04:11:32.458481+0000","last_clean":"2024-03-29T04:11:32.458481+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:11:32.458481+0000","last_undegraded":"2024-03-29T04:11:32.458481+0000","last_fullsized":"2024-03-29T04:11:32.458481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T04:09:03.608815+0000","last_clean_scrub_stamp":"2024-03-29T04:09:03.608815+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:44.905735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":819138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1114112,"data_stored":819138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:11:40.678 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T04:09:03.608815+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=9, tm_sec=3, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=4, tm_min=11, tm_sec=35, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T04:11:40.679 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T04:11:42.196 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:11:42.200 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:00.681 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T04:12:00.794 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.815+0000 7f73099f4640 1 Processor -- start 2024-03-29T04:12:00.795 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.815+0000 7f73099f4640 1 -- start start 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f73099f4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f730415e950 0x7f73040d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f73099f4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73040d30d0 0x7f73040b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f73099f4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73040d36f0 con 0x7f73040d48c0 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f73099f4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f73040d3cb0 con 0x7f730415e950 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f73099f4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f73040d4d00 con 0x7f73040d30d0 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f730415e950 0x7f73040d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:00.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f730415e950 0x7f73040d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50242/0 (socket says 172.21.15.181:50242) 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 -- 172.21.15.181:0/2121416305 learned_addr learned my addr 172.21.15.181:0/2121416305 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 -- 172.21.15.181:0/2121416305 >> v2:172.21.15.187:3300/0 conn(0x7f73040d30d0 msgr2=0x7f73040b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 --2- 172.21.15.181:0/2121416305 >> v2:172.21.15.187:3300/0 conn(0x7f73040d30d0 0x7f73040b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 -- 172.21.15.181:0/2121416305 >> v1:172.21.15.187:6789/0 conn(0x7f73040d48c0 legacy=0x7f73040d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.816+0000 7f7302ffd640 1 -- 172.21.15.181:0/2121416305 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73040d39d0 con 0x7f730415e950 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.817+0000 7f7302ffd640 1 --2- 172.21.15.181:0/2121416305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730415e950 0x7f73040d4380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73040d3e30 tx=0x7f72f402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a74f1119d2b5900 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:00.799 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.817+0000 7f7301ffb640 1 -- 172.21.15.181:0/2121416305 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f402cb00 con 0x7f730415e950 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.817+0000 7f7301ffb640 1 -- 172.21.15.181:0/2121416305 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f402cca0 con 0x7f730415e950 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.817+0000 7f7301ffb640 1 -- 172.21.15.181:0/2121416305 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4034610 con 0x7f730415e950 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 -- 172.21.15.181:0/2121416305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730415e950 msgr2=0x7f73040d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 --2- 172.21.15.181:0/2121416305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730415e950 0x7f73040d4380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73040d3e30 tx=0x7f72f402bf60 comp rx=0 tx=0).stop 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 -- 172.21.15.181:0/2121416305 shutdown_connections 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 --2- 172.21.15.181:0/2121416305 >> v2:172.21.15.187:3300/0 conn(0x7f73040d30d0 0x7f73040b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 --2- 172.21.15.181:0/2121416305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730415e950 0x7f73040d4380 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.800 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 -- 172.21.15.181:0/2121416305 >> 172.21.15.181:0/2121416305 conn(0x7f73040676e0 msgr2=0x7f7304067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:00.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 -- 172.21.15.181:0/2121416305 shutdown_connections 2024-03-29T04:12:00.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.818+0000 7f73099f4640 1 -- 172.21.15.181:0/2121416305 wait complete. 2024-03-29T04:12:00.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.819+0000 7f73099f4640 1 Processor -- start 2024-03-29T04:12:00.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.819+0000 7f73099f4640 1 -- start start 2024-03-29T04:12:00.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73040d30d0 0x7f730414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f730415e950 0x7f7304161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f730415ed30 con 0x7f73040d48c0 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73040b7da0 con 0x7f73040d30d0 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73099f4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f730415eeb0 con 0x7f730415e950 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73027fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:00.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73027fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50246/0 (socket says 172.21.15.181:50246) 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.820+0000 7f73027fc640 1 -- 172.21.15.181:0/2920759502 learned_addr learned my addr 172.21.15.181:0/2920759502 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f7302ffd640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73040d30d0 0x7f730414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73037fe640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f730415e950 0x7f7304161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 -- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f730415e950 msgr2=0x7f7304161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f730415e950 0x7f7304161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 -- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73040d30d0 msgr2=0x7f730414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73040d30d0 0x7f730414d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 -- 172.21.15.181:0/2920759502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730414fca0 con 0x7f73040d48c0 2024-03-29T04:12:00.805 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73027fc640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c410 tx=0x7f72f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f8017070 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f80035f0 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80039e0 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730414ff80 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.821+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7304150540 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.822+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7304075a10 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.823+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f72f801a070 con 0x7f73040d48c0 2024-03-29T04:12:00.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.823+0000 7f72ebfff640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 0x7f72f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:00.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.823+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f72f808a980 con 0x7f73040d48c0 2024-03-29T04:12:00.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.823+0000 7f7302ffd640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 0x7f72f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:00.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.823+0000 7f7302ffd640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 0x7f72f006e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72f4035000 tx=0x7f72f4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:00.807 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.828+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f72f8058a30 con 0x7f73040d48c0 2024-03-29T04:12:00.978 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:00.999+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 --> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73040d42b0 con 0x7f72f006c530 2024-03-29T04:12:00.979 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.000+0000 7f72ebfff640 1 -- 172.21.15.181:0/2920759502 <== mgr.4110 v2:172.21.15.181:6832/4041148452 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7f73040d42b0 con 0x7f72f006c530 2024-03-29T04:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:12:00.980 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2024-03-29T04:12:00.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 msgr2=0x7f72f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:00.981 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 0x7f72f006e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72f4035000 tx=0x7f72f4030000 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 msgr2=0x7f7304150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f72f800c410 tx=0x7f72f800c8d0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 shutdown_connections 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f72f006c530 0x7f72f006e9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f730415e950 0x7f7304161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73040d48c0 0x7f7304150d40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 --2- 172.21.15.181:0/2920759502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73040d30d0 0x7f730414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:00.982 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 >> 172.21.15.181:0/2920759502 conn(0x7f73040676e0 msgr2=0x7f73040cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:00.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 shutdown_connections 2024-03-29T04:12:00.983 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.002+0000 7f73099f4640 1 -- 172.21.15.181:0/2920759502 wait complete. 2024-03-29T04:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-03-29T04:11:59.604476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225560,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749160,"statfs":{"total":773094113280,"available":772863139840,"internally_reserved":0,"allocated":5349376,"data_stored":3010896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002599"},"pg_stats":[{"pgid":"1.0","version":"44'119","reported_seq":227,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-03-29T04:11:41.361845+0000","last_change":"2024-03-29T04:11:41.361890+0000","last_active":"2024-03-29T04:11:41.361845+0000","last_peered":"2024-03-29T04:11:41.361845+0000","last_clean":"2024-03-29T04:11:41.361845+0000","last_became_active":"2024-03-29T04:09:04.723787+0000","last_became_peered":"2024-03-29T04:09:04.723787+0000","last_unstale":"2024-03-29T04:11:41.361845+0000","last_undegraded":"2024-03-29T04:11:41.361845+0000","last_fullsized":"2024-03-29T04:11:41.361845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'119","last_scrub_stamp":"2024-03-29T04:11:41.361766+0000","last_deep_scrub":"44'119","last_deep_scrub_stamp":"2024-03-29T04:11:41.361766+0000","last_clean_scrub_stamp":"2024-03-29T04:11:41.361766+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:44.121181+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1114112,"data_stored":819138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":228770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1114112,"data_stored":819138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T04:12:00.993 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T04:12:01.092 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.113+0000 7f36eb6b1640 1 Processor -- start 2024-03-29T04:12:01.093 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.113+0000 7f36eb6b1640 1 -- start start 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36eb6b1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 0x7f36e415fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36eb6b1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36e41603d0 0x7f36e40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36eb6b1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36e40ce9c0 con 0x7f36e413f0c0 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36eb6b1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f36e4160e30 con 0x7f36e41603d0 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36eb6b1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f36e4160b50 con 0x7f36e406d8f0 2024-03-29T04:12:01.094 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36e9c4e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36e41603d0 0x7f36e40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36e9c4e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f36e41603d0 0x7f36e40cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50258/0 (socket says 172.21.15.181:50258) 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36e9c4e640 1 -- 172.21.15.181:0/369276792 learned_addr learned my addr 172.21.15.181:0/369276792 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.114+0000 7f36e9c4e640 1 -- 172.21.15.181:0/369276792 >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 msgr2=0x7f36e415fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e8c4c640 1 --2- 172.21.15.181:0/369276792 >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 0x7f36e415fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e9c4e640 1 --2- 172.21.15.181:0/369276792 >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 0x7f36e415fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e9c4e640 1 -- 172.21.15.181:0/369276792 >> v1:172.21.15.187:6789/0 conn(0x7f36e413f0c0 legacy=0x7f36e406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:01.095 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e9c4e640 1 -- 172.21.15.181:0/369276792 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e413fa40 con 0x7f36e41603d0 2024-03-29T04:12:01.096 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e8c4c640 1 --2- 172.21.15.181:0/369276792 >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 0x7f36e415fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36e9c4e640 1 --2- 172.21.15.181:0/369276792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e40cfbb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e406e3a0 tx=0x7f36dc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f62bc9c9927179 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36dbfff640 1 -- 172.21.15.181:0/369276792 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc038070 con 0x7f36e41603d0 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36dbfff640 1 -- 172.21.15.181:0/369276792 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc02f050 con 0x7f36e41603d0 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.115+0000 7f36dbfff640 1 -- 172.21.15.181:0/369276792 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc031970 con 0x7f36e41603d0 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 -- 172.21.15.181:0/369276792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 msgr2=0x7f36e40cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.099 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/369276792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e40cfbb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f36e406e3a0 tx=0x7f36dc02b7f0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 -- 172.21.15.181:0/369276792 shutdown_connections 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/369276792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e40cfbb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/369276792 >> v2:172.21.15.187:3300/0 conn(0x7f36e406d8f0 0x7f36e415fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 -- 172.21.15.181:0/369276792 >> 172.21.15.181:0/369276792 conn(0x7f36e40b5b00 msgr2=0x7f36e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 -- 172.21.15.181:0/369276792 shutdown_connections 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.116+0000 7f36eb6b1640 1 -- 172.21.15.181:0/369276792 wait complete. 2024-03-29T04:12:01.100 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.117+0000 7f36eb6b1640 1 Processor -- start 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.117+0000 7f36eb6b1640 1 -- start start 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e406d8f0 0x7f36e413c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 0x7f36e413bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4160840 con 0x7f36e41603d0 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4140000 con 0x7f36e413f0c0 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36eb6b1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36e413f4a0 con 0x7f36e406d8f0 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.101 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50270/0 (socket says 172.21.15.181:50270) 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 -- 172.21.15.181:0/234228222 learned_addr learned my addr 172.21.15.181:0/234228222 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e8c4c640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 0x7f36e413bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e944d640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e406d8f0 0x7f36e413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 -- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e406d8f0 msgr2=0x7f36e413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e406d8f0 0x7f36e413c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 -- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 msgr2=0x7f36e413bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 0x7f36e413bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.102 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e9c4e640 1 -- 172.21.15.181:0/234228222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e41e6ed0 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.118+0000 7f36e8c4c640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 0x7f36e413bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36e9c4e640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f36dc02b830 tx=0x7f36dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc042070 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36dc03d890 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc038040 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41e71b0 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.119+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e4143f30 con 0x7f36e41603d0 2024-03-29T04:12:01.103 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.120+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e4075890 con 0x7f36e41603d0 2024-03-29T04:12:01.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.120+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f36dc045070 con 0x7f36e41603d0 2024-03-29T04:12:01.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.121+0000 7f36d9ffb640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 0x7f36c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.104 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.121+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f36dc0ad3a0 con 0x7f36e41603d0 2024-03-29T04:12:01.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.121+0000 7f36e944d640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 0x7f36c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.121+0000 7f36e944d640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 0x7f36c806e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36e415ebe0 tx=0x7f36e00094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.105 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.125+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36e40ce9c0 con 0x7f36e41603d0 2024-03-29T04:12:01.279 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.300+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f36e406c2c0 con 0x7f36e41603d0 2024-03-29T04:12:01.282 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.303+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f36dc07bb30 con 0x7f36e41603d0 2024-03-29T04:12:01.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.305+0000 7f36d9ffb640 1 -- 172.21.15.181:0/234228222 <== mon.0 v2:172.21.15.181:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f36dc07b430 con 0x7f36e41603d0 2024-03-29T04:12:01.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 msgr2=0x7f36c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 0x7f36c806e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36e415ebe0 tx=0x7f36e00094d0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 msgr2=0x7f36e4071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f36dc02b830 tx=0x7f36dc0046c0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 shutdown_connections 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36e41603d0 0x7f36e4071420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.286 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e413f0c0 0x7f36e413bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f36c806c530 0x7f36c806e9f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 --2- 172.21.15.181:0/234228222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36e406d8f0 0x7f36e413c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 >> 172.21.15.181:0/234228222 conn(0x7f36e40b5b00 msgr2=0x7f36e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:01.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 shutdown_connections 2024-03-29T04:12:01.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.306+0000 7f36eb6b1640 1 -- 172.21.15.181:0/234228222 wait complete. 2024-03-29T04:12:01.296 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T04:12:01.297 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T04:12:01.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.417+0000 7f986608f640 1 Processor -- start 2024-03-29T04:12:01.397 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 -- start start 2024-03-29T04:12:01.399 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f986015b270 0x7f986015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 0x7f98600ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986015b650 con 0x7f9860072280 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 -- 0x7f9860072ab0 con 0x7f986015b270 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f986608f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9860072d90 con 0x7f98600cd0e0 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f985f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f986015b270 0x7f986015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f9856ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9860072280 0x7f98600cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:42494/0 (socket says 172.21.15.181:42494) 2024-03-29T04:12:01.400 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.418+0000 7f9856ffd640 1 -- 172.21.15.181:0/3345017825 learned_addr learned my addr 172.21.15.181:0/3345017825 (peer_addr_for_me v1:172.21.15.181:0/0) 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985f7fe640 1 -- 172.21.15.181:0/3345017825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 msgr2=0x7f98600ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985ffff640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 0x7f98600ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985f7fe640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 0x7f98600ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985f7fe640 1 -- 172.21.15.181:0/3345017825 >> v1:172.21.15.181:6789/0 conn(0x7f9860072280 legacy=0x7f98600cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985effd640 1 -- 172.21.15.181:0/3345017825 <== mon.0 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942687312 0 0) 0x7f986015b650 con 0x7f9860072280 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985f7fe640 1 -- 172.21.15.181:0/3345017825 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860073070 con 0x7f986015b270 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985ffff640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 0x7f98600ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T04:12:01.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.419+0000 7f985f7fe640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f986015b270 0x7f986015f7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f98500097a0 tx=0x7f985002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7f44fbb11abc08 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f985effd640 1 -- 172.21.15.181:0/3345017825 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985002f8a0 con 0x7f986015b270 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f985effd640 1 -- 172.21.15.181:0/3345017825 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f985002fa40 con 0x7f986015b270 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f985effd640 1 -- 172.21.15.181:0/3345017825 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985002fe30 con 0x7f986015b270 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 -- 172.21.15.181:0/3345017825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f986015b270 msgr2=0x7f986015f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f986015b270 0x7f986015f7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f98500097a0 tx=0x7f985002dae0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 -- 172.21.15.181:0/3345017825 shutdown_connections 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98600cd0e0 0x7f98600ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 --2- 172.21.15.181:0/3345017825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f986015b270 0x7f986015f7e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.402 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 -- 172.21.15.181:0/3345017825 >> 172.21.15.181:0/3345017825 conn(0x7f98600b1c90 msgr2=0x7f98600b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:01.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 -- 172.21.15.181:0/3345017825 shutdown_connections 2024-03-29T04:12:01.403 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.420+0000 7f986608f640 1 -- 172.21.15.181:0/3345017825 wait complete. 2024-03-29T04:12:01.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.421+0000 7f986608f640 1 Processor -- start 2024-03-29T04:12:01.405 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 -- start start 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 0x7f986015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f986015b270 0x7f986015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f98600726e0 con 0x7f9860072280 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9860073630 con 0x7f98600cd0e0 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f986608f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9860072860 con 0x7f986015b270 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f985f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f985f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:50288/0 (socket says 172.21.15.181:50288) 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f985f7fe640 1 -- 172.21.15.181:0/1924239050 learned_addr learned my addr 172.21.15.181:0/1924239050 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f9856ffd640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 0x7f986015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.422+0000 7f985ffff640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f986015b270 0x7f986015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 -- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f986015b270 msgr2=0x7f986015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f986015b270 0x7f986015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 -- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 msgr2=0x7f986015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 0x7f986015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.407 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 -- 172.21.15.181:0/1924239050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860144f70 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f9856ffd640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 0x7f986015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985f7fe640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850049070 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9850039400 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850039810 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9860150f40 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.423+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9860151500 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.424+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98600d0d20 con 0x7f9860072280 2024-03-29T04:12:01.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.424+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f98500399b0 con 0x7f9860072280 2024-03-29T04:12:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.425+0000 7f985cff9640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 0x7f984406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T04:12:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.425+0000 7f9856ffd640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 0x7f984406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T04:12:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.425+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f98500af610 con 0x7f9860072280 2024-03-29T04:12:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.425+0000 7f9856ffd640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 0x7f984406e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f98601f3620 tx=0x7f9848007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T04:12:01.409 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.429+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f98600d0d20 con 0x7f9860072280 2024-03-29T04:12:01.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.643+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9860075a10 con 0x7f9860072280 2024-03-29T04:12:01.622 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.643+0000 7f985cff9640 1 -- 172.21.15.181:0/1924239050 <== mon.0 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3878 (secure 0 0 0) 0x7f9860144f70 con 0x7f9860072280 2024-03-29T04:12:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-03-29T04:12:01.623 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1142387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1142387778},{"type":"v1","addr":"172.21.15.187:6834","nonce":1142387778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2552036906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2552036906},{"type":"v1","addr":"172.21.15.187:6835","nonce":2552036906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3674673280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3674673280},{"type":"v1","addr":"172.21.15.181:6836","nonce":3674673280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5799,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/399672739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":399672739},{"type":"v1","addr":"172.21.15.181:6837","nonce":399672739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-03-29T04:12:01.623 INFO:teuthology.orchestra.run.smithi181.stderr:dumped fsmap epoch 20 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 msgr2=0x7f984406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 0x7f984406e9f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f98601f3620 tx=0x7f9848007c10 comp rx=0 tx=0).stop 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 msgr2=0x7f9860144a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9850000c00 tx=0x7f9850004730 comp rx=0 tx=0).stop 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 shutdown_connections 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:6832/4041148452,v1:172.21.15.181:6833/4041148452] conn(0x7f984406c530 0x7f984406e9f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f986015b270 0x7f986015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.625 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98600cd0e0 0x7f986015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 --2- 172.21.15.181:0/1924239050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9860072280 0x7f9860144a30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T04:12:01.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 >> 172.21.15.181:0/1924239050 conn(0x7f98600b1c90 msgr2=0x7f98600cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T04:12:01.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 shutdown_connections 2024-03-29T04:12:01.626 INFO:teuthology.orchestra.run.smithi181.stderr:2024-03-29T04:12:01.645+0000 7f986608f640 1 -- 172.21.15.181:0/1924239050 wait complete. 2024-03-29T04:12:01.636 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T04:12:01.636 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T04:12:01.636 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T04:12:01.636 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.645 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T04:12:01.645 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T04:12:01.645 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.694 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T04:12:01.694 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T04:12:01.694 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.706 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T04:12:01.706 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T04:12:01.706 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.758 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T04:12:01.759 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T04:12:01.759 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T04:12:01.759 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.791 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T04:12:01.791 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T04:12:01.791 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.859 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T04:12:01.860 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T04:12:01.860 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:01.936 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T04:12:01.936 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T04:12:01.936 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.001 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T04:12:02.001 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T04:12:02.001 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.035 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T04:12:02.036 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T04:12:02.036 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.108 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T04:12:02.109 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T04:12:02.109 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.179 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T04:12:02.180 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T04:12:02.180 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.252 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T04:12:02.252 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T04:12:02.252 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T04:12:02.252 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.278 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T04:12:02.278 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T04:12:02.278 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.318 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T04:12:02.319 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T04:12:02.319 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T04:12:02.319 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.331 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T04:12:02.331 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T04:12:02.331 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.371 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T04:12:02.372 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T04:12:02.372 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T04:12:02.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-29T04:12:02.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-29T04:12:02.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-29T04:12:02.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-29T04:12:02.425 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T04:12:02.425 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T04:12:02.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-29T04:12:02.463 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:12:02.464 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T04:12:02.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:12:02.708 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T04:12:02.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:12:02.801 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T04:12:02.859 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:12:02.860 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T04:12:02.917 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:12:02.918 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T04:12:03.035 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:12:03.035 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T04:12:03.107 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:12:03.107 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T04:12:03.164 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:12:03.164 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T04:12:03.221 INFO:tasks.ceph:Archiving mon data... 2024-03-29T04:12:03.223 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/data/mon.a.tgz 2024-03-29T04:12:03.223 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-03-29T04:12:03.238 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.CgeFzMlMCp 2024-03-29T04:12:03.238 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.CgeFzMlMCp 2024-03-29T04:12:03.363 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.CgeFzMlMCp 2024-03-29T04:12:03.448 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.CgeFzMlMCp is 320KB 2024-03-29T04:12:03.546 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.CgeFzMlMCp 2024-03-29T04:12:03.561 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/data/mon.b.tgz 2024-03-29T04:12:03.561 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-03-29T04:12:03.578 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.SQKYAYc79k 2024-03-29T04:12:03.578 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.SQKYAYc79k 2024-03-29T04:12:03.747 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.SQKYAYc79k 2024-03-29T04:12:03.833 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.SQKYAYc79k is 423KB 2024-03-29T04:12:03.931 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.SQKYAYc79k 2024-03-29T04:12:03.945 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/data/mon.c.tgz 2024-03-29T04:12:03.945 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-03-29T04:12:03.999 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.T27XdvGCQp 2024-03-29T04:12:04.000 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.T27XdvGCQp 2024-03-29T04:12:04.145 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.T27XdvGCQp 2024-03-29T04:12:04.228 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.T27XdvGCQp is 423KB 2024-03-29T04:12:04.327 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.T27XdvGCQp 2024-03-29T04:12:04.341 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T04:12:04.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:12:04.344 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T04:12:04.430 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:12:04.462 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-29T04:12:04.463 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T04:12:04.464 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi181/crash 2024-03-29T04:12:04.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:12:04.499 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi187/crash 2024-03-29T04:12:04.500 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T04:12:04.529 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T04:12:04.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:04.544 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:04.650 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:04.678 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T04:12:04.705 INFO:tasks.ceph:Compressing logs... 2024-03-29T04:12:04.705 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:12:04.722 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-29T04:12:04.746 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44099.log 2024-03-29T04:12:04.791 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-03-29T04:12:04.791 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37558.log 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37398.log 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37398.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37558.log.gz 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44948.log 2024-03-29T04:12:04.792 INFO:teuthology.orchestra.run.smithi187.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37398.log.gz 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37478.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.44948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44948.log.gz 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43954.log 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.43954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 0.0% 2024-03-29T04:12:04.793 INFO:teuthology.orchestra.run.smithi187.stderr: -- replaced with /var/log/ceph/ceph-client.1.43954.log.gz 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.37438.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37598.log 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37598.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:12:04.794 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37598.log.gz 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37678.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44351.log 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37638.log: /var/log/ceph/ceph-mon.b.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-03-29T04:12:04.795 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-29T04:12:04.796 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.44351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44351.log.gz 2024-03-29T04:12:04.796 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-29T04:12:04.796 INFO:teuthology.orchestra.run.smithi187.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-29T04:12:04.796 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-29T04:12:04.811 INFO:teuthology.orchestra.run.smithi187.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-29T04:12:04.871 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47578.log 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47410.log 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47578.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.46054.log 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44099.log.gz 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47578.log.gz 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-03-29T04:12:04.872 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47410.log.gz 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56080.log 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56080.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 88.1% 2024-03-29T04:12:04.873 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56080.log.gz 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49768.log 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43673.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz /var/log/ceph/ceph-client.admin.52208.log 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9%/var/log/ceph/ceph-client.admin.49768.log: -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51752.log 2024-03-29T04:12:04.874 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49768.log.gz 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1%/var/log/ceph/ceph-client.admin.51752.log: -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51752.log.gz 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44988.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.45712.log 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-03-29T04:12:04.875 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50594.log 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50594.log: gzip 88.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45712.log.gz 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr: /var/log/ceph/ceph-client.admin.53065.log 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50594.log.gz 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46393.log 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56332.log 2024-03-29T04:12:04.876 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44904.log 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53065.log.gz 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46393.log.gz 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56332.log.gz 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56206.log 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46053.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44904.log.gz 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54905.log 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56206.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-03-29T04:12:04.877 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54349.log 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56206.log.gz 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51260.log 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54349.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54905.log.gz 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50006.log 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54349.log.gz 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51260.log.gz 2024-03-29T04:12:04.878 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44015.log 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43486.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50006.log.gz 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49264.log 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51447.log 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44015.log.gz 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51447.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.49264.log.gz -5 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-03-29T04:12:04.879 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51447.log.gz 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52729.log 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49894.log 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-03-29T04:12:04.880 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1%/var/log/ceph/ceph-client.admin.49894.log: -- replaced with /var/log/ceph/ceph-client.admin.52729.log.gz 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45030.log 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49894.log.gz 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55996.log 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45030.log.gz 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54475.log 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55996.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-03-29T04:12:04.881 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55996.log.gz 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54475.log.gz 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51874.log 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz -5 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-03-29T04:12:04.882 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51874.log.gz 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46387.log 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45586.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-03-29T04:12:04.883 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46387.log.gz 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47536.log: /var/log/ceph/ceph-client.admin.51794.log 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46062.log 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56248.log 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51794.log.gz 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-03-29T04:12:04.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56248.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46062.log.gz 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56248.log.gz 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37541.log 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55073.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2024-03-29T04:12:04.885 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.37541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52420.log 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.37541.log.gz 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52420.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52420.log.gz 2024-03-29T04:12:04.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54989.log 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56374.log 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54989.log.gz 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45214.log 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-03-29T04:12:04.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45214.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56374.log.gz 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31512.log 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45214.log.gz 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.37804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.tmp-client.admin.31512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31512.log.gz 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50462.log 2024-03-29T04:12:04.888 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50132.log 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50462.log: gzip 89.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50132.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.50462.log.gz -5 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50132.log.gz 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45256.log 2024-03-29T04:12:04.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49810.log 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45256.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47620.log 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45256.log.gz 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52645.log 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47620.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49810.log.gz 2024-03-29T04:12:04.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52645.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47620.log.gz /var/log/ceph/ceph-client.admin.48928.log 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52645.log.gz 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47182.log 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip/var/log/ceph/ceph-client.admin.48928.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53570.log 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2024-03-29T04:12:04.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54947.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47182.log.gz 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46710.log 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.audit.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53570.log.gz 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55157.log 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51302.log 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55157.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46710.log.gz 2024-03-29T04:12:04.892 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55157.log.gz 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51302.log.gz 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53712.log 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50924.log: /var/log/ceph/ceph-client.admin.45502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2024-03-29T04:12:04.893 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4%/var/log/ceph/ceph-client.admin.53712.log: -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55470.log 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56642.log 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55470.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53712.log.gz 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55470.log.gz 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 2024-03-29T04:12:04.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55954.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.56642.log.gz 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz/var/log/ceph/ceph-client.admin.54863.log: 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51626.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49600.log 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5%/var/log/ceph/ceph-client.admin.47224.log: -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2024-03-29T04:12:04.895 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51176.log 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49600.log.gz 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51176.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46930.log 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51176.log.gz 2024-03-29T04:12:04.896 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52939.log 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46930.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52939.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46930.log.gz 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37896.log 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52939.log.gz 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46714.log 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.37896.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz -5 2024-03-29T04:12:04.897 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52981.log 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46714.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37896.log.gz 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45670.log 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46714.log.gz 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52376.log 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45670.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52981.log.gz 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46057.log 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45670.log.gz 2024-03-29T04:12:04.898 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46736.log 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52376.log.gz 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55031.log 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46057.log.gz 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55031.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46736.log.gz 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50804.log 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55031.log.gz 2024-03-29T04:12:04.899 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53173.log 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50804.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz /var/log/ceph/ceph-client.admin.56600.log 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53173.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50804.log.gz 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49474.log 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53173.log.gz 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56600.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56600.log.gz 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52082.log 2024-03-29T04:12:04.900 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.1.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49474.log.gz 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52687.log 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49684.log 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52687.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52082.log.gz 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52687.log.gz 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52771.log 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49684.log: /var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51386.log 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49684.log.gz 2024-03-29T04:12:04.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52771.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51386.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52771.log.gz 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43845.log 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51386.log.gz 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49390.log 2024-03-29T04:12:04.902 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51008.log: /var/log/ceph/ceph-client.admin.43845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51997.log 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43845.log.gz 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54517.log 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49390.log.gz 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51997.log: /var/log/ceph/ceph-client.admin.54517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51997.log.gz 2024-03-29T04:12:04.903 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48970.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54517.log.gz 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49432.log 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46712.log: /var/log/ceph/ceph-client.admin.43547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54637.log 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49432.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46394.log 2024-03-29T04:12:04.904 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49432.log.gz 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47014.log 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54637.log: /var/log/ceph/ceph-client.admin.46394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45172.log 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54637.log.gz 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47014.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46394.log.gz 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51583.log 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47014.log.gz 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44057.log 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45172.log: /var/log/ceph/ceph-client.admin.51583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-03-29T04:12:04.905 INFO:teuthology.orchestra.run.smithi181.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45172.log.gz 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44057.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51583.log.gz 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45880.log 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52039.log 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9%/var/log/ceph/ceph-client.admin.50048.log: -- replaced with /var/log/ceph/ceph-client.admin.44057.log.gz 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45880.log.gz 2024-03-29T04:12:04.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52250.log 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52039.log: /var/log/ceph/ceph-client.admin.56122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55386.log 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52039.log.gz 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52250.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51134.log 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52250.log.gz 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50636.log 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47494.log 2024-03-29T04:12:04.907 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51134.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55386.log.gz 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50636.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51134.log.gz 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45072.log 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50636.log.gz 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47494.log: /var/log/ceph/ceph-client.admin.45072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45754.log 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47494.log.gz 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.3.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45072.log.gz 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46058.log 2024-03-29T04:12:04.908 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44712.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45754.log: /var/log/ceph/ceph-client.admin.46058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.48584.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45754.log.gz 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44712.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46058.log.gz 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50678.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.2.48584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51668.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.48584.log.gz 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51668.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2024-03-29T04:12:04.909 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50678.log.gz 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8%/var/log/ceph/ceph-client.admin.52166.log: -- replaced with /var/log/ceph/ceph-client.admin.51668.log.gz 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54391.log 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr:gzip 92.7% -- replaced with /var/log/ceph/ceph-client.admin.44712.log.gz 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51050.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46397.log 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50720.log 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2024-03-29T04:12:04.910 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46397.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54391.log.gz -5 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50720.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46397.log.gz 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43887.log 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50720.log.gz 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53862.log 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50882.log 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2024-03-29T04:12:04.911 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53862.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43887.log.gz 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52599.log 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53862.log.gz 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43267.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50882.log.gz 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53361.log 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46395.log 2024-03-29T04:12:04.912 INFO:teuthology.orchestra.run.smithi181.stderr: 89.1%/var/log/ceph/ceph-client.admin.52599.log: -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.53361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53361.log.gz 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50762.log 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52599.log.gz 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46395.log: /var/log/ceph/ceph-client.admin.50762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47140.log 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46395.log.gz 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50762.log.gz 2024-03-29T04:12:04.913 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51710.log 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47140.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45114.log 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47140.log.gz 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45114.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51710.log.gz 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45114.log.gz 2024-03-29T04:12:04.914 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56164.log 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.y.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44291.log 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52464.log 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56416.log 2024-03-29T04:12:04.915 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56164.log.gz 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52334.log 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52464.log.gz 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56416.log: /var/log/ceph/ceph-client.admin.52334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54595.log 88.4% 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56416.log.gz 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46390.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52334.log.gz 2024-03-29T04:12:04.916 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54595.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49642.log 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54595.log.gz 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44141.log 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr: -5/var/log/ceph/ceph-client.admin.49642.log: --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44141.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49642.log.gz 2024-03-29T04:12:04.917 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44141.log.gz 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50336.log 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49138.log: -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44225.log 2024-03-29T04:12:04.918 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55241.log 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50336.log.gz 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48718.log 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.44225.log: gzip/var/log/ceph/ceph-client.admin.55241.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51955.log 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48718.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55241.log.gz 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-03-29T04:12:04.919 INFO:teuthology.orchestra.run.smithi181.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44225.log.gz 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46711.log 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51955.log: /var/log/ceph/ceph-client.admin.49096.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51955.log.gz 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46711.log: gzip 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54433.log 2024-03-29T04:12:04.920 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2%/var/log/ceph/ceph-client.admin.47098.log: -- replaced with /var/log/ceph/ceph-client.admin.46711.log.gz 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55301.log 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55744.log 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55301.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50420.log 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54433.log.gz 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55301.log.gz 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56290.log 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50420.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55744.log.gz 2024-03-29T04:12:04.921 INFO:teuthology.orchestra.run.smithi181.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52897.log 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56290.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50420.log.gz 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56038.log 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56290.log.gz 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:gzip/var/log/ceph/ceph-client.admin.52897.log: -5 --verbose -- /var/log/ceph/ceph-client.0.48145.log 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56038.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52897.log.gz 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56038.log.gz 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr: -- /var/log/ceph/ceph-client.admin.52124.log 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.0.48145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48145.log.gz 2024-03-29T04:12:04.922 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52124.log.gz 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47368.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:gzip/var/log/ceph/ceph-client.admin.54821.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55199.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47368.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56500.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47368.log.gz 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51344.log 2024-03-29T04:12:04.923 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55199.log.gz 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51344.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56500.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51344.log.gz 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.54721.log: /var/log/ceph/ceph-client.admin.49348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46398.log 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48802.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-03-29T04:12:04.924 INFO:teuthology.orchestra.run.smithi181.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49726.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46398.log: gzip -5 88.3% --verbose -- /var/log/ceph/ceph-client.admin.43931.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46398.log.gz 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49726.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49726.log.gz/var/log/ceph/ceph-client.admin.43931.log: 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46389.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43931.log.gz 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56458.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55828.log 2024-03-29T04:12:04.925 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46389.log.gz 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56458.log: gzip -5 --verbose -- 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56458.log.gz 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr: /var/log/ceph/ceph-osd.2.log 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55828.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55828.log.gz 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr: /var/log/ceph/ceph-mds.c.log 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55912.log 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49558.log 2024-03-29T04:12:04.926 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55912.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55912.log.gz 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45796.log 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45796.log: 88.5% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45796.log.gz 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49558.log.gz 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.48280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51541.log 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55344.log 2024-03-29T04:12:04.927 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51541.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51541.log.gz 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49852.log 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46060.log: /var/log/ceph/ceph-client.admin.55344.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55344.log.gz 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45382.log 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.49852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47826.log 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49852.log.gz 2024-03-29T04:12:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50504.log 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47826.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47826.log.gz 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43363.log 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55660.log 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50504.log.gz 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52855.log 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55660.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43363.log.gz 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55660.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-03-29T04:12:04.929 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53493.log 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52855.log: /var/log/ceph/ceph-client.admin.43759.log: 89.4% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52855.log.gz 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51092.log 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47662.log 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53493.log.gz 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.51092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47326.log 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47662.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51092.log.gz 2024-03-29T04:12:04.930 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53023.log 2024-03-29T04:12:04.931 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47326.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47662.log.gz 2024-03-29T04:12:04.933 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47326.log.gz 2024-03-29T04:12:04.933 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2024-03-29T04:12:04.933 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.53023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50966.log 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53023.log.gz 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46594.log 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50966.log.gz 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-03-29T04:12:04.934 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46594.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46594.log.gz 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56560.log 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.45298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.56560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55115.log 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.50174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56560.log.gz 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52292.log 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2024-03-29T04:12:04.935 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.55115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.52292.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55115.log.gz 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52292.log.gz 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.50378.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.47056.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-03-29T04:12:04.936 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46709.log 2024-03-29T04:12:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43175.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-03-29T04:12:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43973.log 2024-03-29T04:12:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.46709.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46709.log.gz 2024-03-29T04:12:04.937 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.43973.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43973.log.gz 2024-03-29T04:12:04.937 INFO:teuthology.orchestra.run.smithi181.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-29T04:12:04.938 INFO:teuthology.orchestra.run.smithi187.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-29T04:12:04.938 INFO:teuthology.orchestra.run.smithi187.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-29T04:12:04.974 INFO:teuthology.orchestra.run.smithi181.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-29T04:12:05.080 INFO:teuthology.orchestra.run.smithi181.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-29T04:12:05.436 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-29T04:12:05.472 INFO:teuthology.orchestra.run.smithi187.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-29T04:12:05.495 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-29T04:12:05.527 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-29T04:12:05.529 INFO:teuthology.orchestra.run.smithi187.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-29T04:12:05.531 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-03-29T04:12:05.531 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m0.772s 2024-03-29T04:12:05.531 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m3.032s 2024-03-29T04:12:05.531 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.145s 2024-03-29T04:12:05.542 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-29T04:12:05.561 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-29T04:12:05.658 INFO:teuthology.orchestra.run.smithi181.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-29T04:12:05.660 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-03-29T04:12:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m0.926s 2024-03-29T04:12:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m3.589s 2024-03-29T04:12:05.660 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.270s 2024-03-29T04:12:05.660 INFO:tasks.ceph:Archiving logs... 2024-03-29T04:12:05.660 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi181/log 2024-03-29T04:12:05.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:12:06.421 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi187/log 2024-03-29T04:12:06.422 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T04:12:06.901 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T04:12:06.911 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T04:12:06.911 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:12:06.914 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T04:12:07.009 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T04:12:07.018 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T04:12:07.019 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:12:07.021 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T04:12:07.033 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:12:07.036 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-03-29T04:12:07.037 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:12:07.037 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-03-29T04:12:07.037 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +77us[ +79us] +/- 79ms 2024-03-29T04:12:07.038 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -1284us[-1282us] +/- 70ms 2024-03-29T04:12:07.038 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv03.front.sepia.ceph.com 3 6 377 35 +1166us[+1168us] +/- 67ms 2024-03-29T04:12:07.038 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:12:07.040 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T04:12:07.040 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-03-29T04:12:07.040 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +72us[ +75us] +/- 79ms 2024-03-29T04:12:07.040 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -1286us[-1286us] +/- 70ms 2024-03-29T04:12:07.040 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv03.front.sepia.ceph.com 3 6 377 36 +1171us[+1173us] +/- 67ms 2024-03-29T04:12:07.041 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T04:12:07.041 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T04:12:07.051 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T04:12:07.052 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T04:12:07.060 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:12:07.153 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T04:12:07.224 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:12:07.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:07.254 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-29T04:12:07.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:07.285 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T04:12:07.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T04:12:07.304 INFO:teuthology.task.internal:Duration was 627.730701 seconds 2024-03-29T04:12:07.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T04:12:07.315 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T04:12:07.315 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:12:07.319 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T04:12:07.360 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:12:07.369 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T04:12:07.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-03-29T04:12:07.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-03-29T04:12:07.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-29T04:12:07.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-29T04:12:07.700 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T04:12:07.700 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-03-29T04:12:07.700 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:12:07.760 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-03-29T04:12:07.760 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T04:12:07.789 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T04:12:07.789 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:12:07.804 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T04:12:07.858 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T04:12:07.859 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:12:07.872 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T04:12:08.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T04:12:08.012 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T04:12:08.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:12:08.039 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T04:12:08.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T04:12:08.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:12:08.082 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:12:08.105 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-03-29T04:12:08.132 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-03-29T04:12:08.149 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:12:08.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:08.178 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T04:12:08.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T04:12:08.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T04:12:08.218 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T04:12:08.219 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi181 2024-03-29T04:12:08.219 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:12:08.257 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082/remote/smithi187 2024-03-29T04:12:08.257 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T04:12:08.293 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T04:12:08.293 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:12:08.295 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T04:12:08.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T04:12:08.354 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T04:12:08.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T04:12:08.364 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T04:12:08.365 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:12:08.367 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T04:12:08.379 INFO:teuthology.orchestra.run.smithi181.stdout: 265544 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:12 /home/ubuntu/cephtest 2024-03-29T04:12:08.396 INFO:teuthology.orchestra.run.smithi187.stdout: 265694 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 04:12 /home/ubuntu/cephtest 2024-03-29T04:12:08.397 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T04:12:08.407 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T04:12:08.429 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T04:12:08.458 DEBUG:teuthology.nuke:shortname: smithi181 2024-03-29T04:12:08.458 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:12:08.486 DEBUG:teuthology.nuke:shortname: smithi187 2024-03-29T04:12:08.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T04:12:08.490 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:40.831617', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBX/vqhpoVv7xQF9OaCd4qKW/a5NFdu1jYL3M9RLEAQtc36m1FhNG7RquWYaxP94P98XjDG4nPnlwf63rzrhEg='} 2024-03-29T04:12:08.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_02:01:30-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629082', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 03:53:40.833252', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVLz0WdmvAmWumf0aY3sfydbO2fS/7Cp/rwFGjG55ZgtF/39aOqwD8YdyUV7v2Ca0C0PHSlSZxhsByEx8VSW4g='} 2024-03-29T04:12:08.546 INFO:teuthology.orchestra.console.smithi181:Power off 2024-03-29T04:12:08.546 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:12:08.573 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2024-03-29T04:12:08.573 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:08.589 INFO:teuthology.orchestra.console.smithi187:Power off 2024-03-29T04:12:08.589 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T04:12:08.599 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-03-29T04:12:08.612 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-03-29T04:12:08.612 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:08.636 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-03-29T04:12:12.600 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:12.637 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:12.711 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-03-29T04:12:12.748 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-03-29T04:12:16.714 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:16.748 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:16.826 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2024-03-29T04:12:16.859 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-03-29T04:12:16.959 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-03-29T04:12:20.830 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T04:12:20.940 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2024-03-29T04:12:21.041 INFO:teuthology.orchestra.console.smithi181:Power off completed 2024-03-29T04:12:21.146 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} duration: 627.730700969696 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c7d73a7385aa414093fa018315f76f52 status: fail success: false 2024-03-29T04:12:21.146 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T04:12:21.226 INFO:teuthology.run:FAIL